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REMARKS 

Applicant appreciates the time taken by the Examiner to review Applicant's present 
application. This application has been carefully reviewed in light of the Official Action mailed 
February 7, 2007. Claims 1-43 remain pending in this application. Applicant respectfully 
requests reconsideration and favorable action in this case. 



Rejections under 35 U.S.C. S 102 

The Examiner has maintained the rejections of claims 1-43 under 35 U.S.C. §1 02(e) as 
being anticipated by U.S. Publication No. 20040107319 ("D'Orto"). 

The effective date of a U.S. patent U.S. patent application publication, or international 
application publication under PCT Article 21(2) is the earlier of its publication date or date that it 
is effective as a reference under 35 U.S.C. § 102(e). See 37 C.F.R 1 .131 . Applicant notes that 
the effective date on which the D'Orto reference becomes available as prior art is the filing date 
of U.S. Patent Application No. 10/308,459 corresponding to D'Orto. Namely, December 3, 
2002. Applicant respectfully submits that the subject matter of the present application was 
invented before the December 3, 2002 effective date of the D'Orto reference. 

Conleth O'Connell, Mark Scheevel and N. Isaac Rajkumar (employees of Vignette 
Corporation) conceived of the invention of the present application prior to the filing date of the 
D'Orto patent application (December 3, 2002). More specifically, the invention of the present 
application was conceived of on or before November 20, 2002, as evidenced by the Cache on 
Delivery/Cache Manager Design Document ("Cache Document") composed on or before 
November 20, 2002. The Cache Document demonstrates a conception of the invention 
described and claimed in the present application. See, Declarations Under 37 C.F.R. 1.131 
(the "O'Connell Declaration", "Scheevel Declaration" and "Rajkumar Declaration") 1ffl1-5, 
appended hereto as, respectively, Exhibit 1 , 2 and 3. 
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While the Examiner states in the Office Action that "Applicant must give a clear 
explanation of the exhibit pointing out exactly what facts are established and relied on by 
applicant". Applicant is not aware of any such requirement and respectfully requests that if the 
Examiner knows the source of such requirements with respect to declarations under C.F.R. 
1.131 that the Examiner point out the source of these requirements to the Applicant. 
Additionally, however, the Applicant notes that at least Pages 21-42 of Exhibit A of the 
O'Connell Declaration, Scheevel Declaration and Rajkumar Declaration support the limitations 
of the claims currently pending in the current application. 

Furthermore, the invention of the present application was subsequently reduced to 
practice by the filing, on December 13, 2002, of U.S. Provisional Application 60/433,408 on 
which the present application is based and diligence was maintained between the conception of 
the invention and the filing of this provisional patent application. 

The Examiner states that a "conception of an invention... has no effect... UNLESS THE 
INVENTOR FOLLOWS IT WITH REASONABLE DILIGENCE BY SOME OTHER ACT such as 
an actual reduction to practice or filing an application for a patent." Applicant notes that U.S. 
Provisional Application 60/433,408 on which the present application is based was filed on 
December 13, 2002. As the Examiner noted, the critical period in which diligence must be 
shown begins just prior to the effective data of the reference (here the December 3, 2002 filing 
date of the D'Orto reference) and ends with the data of a reduction to practice (here the 
December 13, 2002 filing date of U.S. Provisional Application 60/433,408 on which the present 
application is based). Thus, a time period of approximately 1 1 days elapsed between the date 
of December 2, 2002 Qust prior to the effective data of the D'Orto reference: December 3, 
2002) and the filing date of U.S. Provisional Application 60/433,408: December 13, 2002. 
Applicant respectfully submits that a time period of 1 1 days exhibits diligence. In fact, in the 
case of Broos v. Barton the court explicitly stated that a time period of 1 1 days between 
conception and a reduction to practice consisting of the filing of a patent application was 
sufficiently diligent. (See, Broos v. Barton , 142 F.2d 690) Applicant submits that this fact 
combined with the statements in the O'Connell Declaration, Scheevel Declaration and 
Rajkumar Declaration are sufficient evidence of diligence. 

As the subject matter of the present application was conceived on or before 
November 20, 2002 (as shown by the O'Connell Declaration, Scheevel Declaration and 
Rajkumar Declaration), subsequently reduced practice by the filing of a provisional 
patent application on December 13, 2002 and diligence was maintained at least 
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between the dates of December 2, 2002 and December 13, 2002; Applicant respectfully 
submits that the subject matter of the present application was invented prior to 
December 3, 2002, the effective data on which the D'Orto reference is available as prior 
art under 35 U.S.C. § 102(e). 

Consequently, Applicant respectfully submits that the D'Orto reference is not 
available as prior art under 35 U.S.C. § 102(e) and respectfully requests the withdrawal 
of the rejection of Claims 1-43 over the D'Orto reference. 



Applicant has now made an earnest attempt to place this case in condition for 
allowance. Other than as explicitly set forth above, this reply does not include an acquiescence 
to statements, assertions, assumptions, conclusions, or any combination thereof in the Office 
Action. For the foregoing reasons and for other reasons clearly apparent, Applicant respectfully 
requests full allowance of Claims 1-43. The Examiner is invited to telephone the undersigned 
at the number listed below for prompt action in the event any issues remain. 

An extension of 3 months is requested and a Notification of Extension of Time Under 37 
C.F.R. § 1.136 with the appropriate fee is enclosed herewith. 

The Director of the U.S. Patent and Trademark Office is hereby authorized to charge 
any fees or credit any overpayments to Deposit Account No. 50-3183 of Sprinkle IP Law Group. 



CONCLUSION 



Respectfully submitted, 




Ari G. Akmal 
Reg. No. 51,388 




1301 W. 25 th Street, Suite 408 
Austin, TX 78705 
Tel. (512)637-9220 
Fax. (512) 371-9088 
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EXHIBIT 1 




HE UNITED STATES PATENT AND TRADEMARK OFFICE 



Declaration Under 37 C.F.R. 1.131 



Atty. Docket No. 



Applicant 


vivarv ipqu-z 


O'Connell, et al 




Application Number 


Date Filed 


10/733,742 


12/11/2003 


Title 




Method and System for Automatic Cache 


Management 




Group Art Unit 


Examiner 


2186 


Patel, Hetul B. 


Confirmation Number: 




8405 





Certification linger 37 C.F.R. Stf m 

I hereby certify that thfs document is being deposited with the 
United States Postal Service with sufficient postage as Express 
Mail No. EV9643411BBUS in an envelope addressed to: 
Commissioner for Patents, P.O. Box 1450, Alexandria VA 
22312-1450 on Januar y 10 . 2007. 

Signature 
Printed Name 



I, Conleth O'Connell, am an original inventor of the invention described and 
claimed in the above-referenced patent application. 

The invention claimed in the above-referenced patent application was conceived 
at least as early as November 20, 2002. 

I participated In the drafting of the Cache on Delivery/Cache Manager Design 
Document ( Cache Document"), attached hereto as Exhibit A, on or before 
November 20,2002. 

The Summary attached hereto as Exhibit A demonstrates a conception of the 
nvention described and claimed in the above-referenced patent application at 
least as early as the date on the Cache Document: November 20, 2002. 

Due diligence was maintained with respect to the invention described and 
Sf'" above-referenced patent application from at least November 20, 
2002 to he filing on December 13, 2002, of the provisional patent application to 
which priority is claimed. 
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Declarant acknowledges that willful false statements and the like are punishable by fine or 
imprisonment, or both (18 U.S.C. 1001) and may jeopardize the validity of the application 
or any patent issuing thereon. 



I, Conleth O'Connell, aver that all statements made of my own knowledge are true and all 
statements made on information and belief are believed to be true. 




Date: Z/ tteg lOt0> 
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I VIGNETTE 



Cache On Delivery (COD) 
Cache Manager (CM) 
Design Document 



Vignette Corporation ' ' 
Two Barton Skyway 
1601 South .MoPac Expressway ' 
• Austin, T3f 78746 • 

Author David Caldwell 

Version 17 . 
-November 20; 2002 



CONFIDENTIAL 

Tins docmbeDt contains confidential and proprietarv inform.,,^ j 
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| Versio 
0.1 
0.2 


n |Date 
1/10/02 
3/25/02 


Author 
_, David Caldwell 
David Caldwell 


Remarks ~~ • : J 
Initial version. 

Placement protocol and placement message handler sections" 

are fairly solid. ' They are reviewed and feedback is 
■ incorporated. Other sections, beware. 


0.3 

• 


5/14/02. 


David Caldwell 
1 - 


Added instance state diagram. 

Added note to iegenerate:iast-accessed-interval about 

possible WS in memory caching problem. 
Renamed physical cache section to cache manager and 

• folded the physical cache's responsibilities into the cache 

manager (since they're 1-1). . 
Added closed issues; moved 1 open issue to closed. 
Reworked placementMsgHandler pursuant to closed issues. 
Added instance deleter 8c .associated.config vara.. 
Reworke^lacement-msg, placementMsgHandler, &. 

associated diagrams with new include nnH nrtirm ' 

schemes, . 

Reject template-placements for app-server'-only if cache is ■ 
ws-only. . 


0.4 


5/15/02 


David Caldwell 


Added concurrency architecture. . 

Removed placement rejection based on appServerOnly flag.- 

Added .setting of expirationTimestamp, 

Rem^vedRAG^GBNERATOR TYPBand 
."WEB_jSERVER TYPB confie vara 


0.5- 1 


5/23/02 . 


David Caldwell 


Added POOL.SIZB config yar. 

Renamed rcdDeployCount to recoidStateGuid. 

Changed placemeritMsgHandler's method of detecting 

concurrent placements from "instance insert duplicate 
row" to "file locked?'' - ■ . • 

Added sections: AM-CM metadata-msg,- A2il-CM record- 
udpate-msg, MetadataMsgHandTer, and 
PeriodicBxpiration. • 


0.6 

[ 0.7 jg 


5/31/02 j'l 
V31A)2 [r 


David Caldwell . 

j 
j 

' . 1 
/ 
B 

>avid Caldwell J A 


T.dded onen issue" flMrmo WoLoomorT . , 

u i~" iosuo. iwjcanng weDserverJLooxup table when 

WS path to PG path mapping changes. 
Kdded clear-cache-msg and handler sections. 
\ddeddgld. ' 

Maintain TemplateHash in TemplateState table, 
tdded ModulelnternalValue. 
Lemoved timezone from expire-period. 
idded RecordUpdateMsgHandler 
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0.8 


■ 6/10/02 

e 

... 


David Caldwell 


Added Regenerator. Changed PlacementMsgHandler to: 

• Handle regen placements 

• Use higher-level Module Values metadata class 
Refined MetadataMsgHandler.updateTemplateMetadata. 
nuucu op^ 11 issue, isxpioaeo deployment ana CDF external" 

.to .war file deployment order "problem. 
Added periodicExpirationTimestamp processing. 
Added HOST, PORT, KBGENERATEJiOST, &' 

REGHNERATP POPTr^Yvfiw^a*^ 

. Moved message receivers under generic fhunework. 

I Put SOIQe Stuff in MnTHtnrirur e#»f*f?/*vr\ 




6/12/02 . 


David Caldwell 


Conform to new ModuIeParams metadata-msg structure: 
Handle placement-msg in 1st part of hup msg rather than 

preamble + subsequent part lengths aie now Content- . . 
. length headers in each part. 


0.10 


6/13/02 


David Caldwell 


Minor formatting changes. 

Increased CLEANUP J)BLAY to 5 minutes. - 


0.11 


7/11/02 


David Caldwell 

.1 

J 

• ■ ' r 
r 

• c 

1 E 
f C 


Corrected "PG Placement message to CM"~diagram in* ■ 
section 4.3. * " ' ! 

Added references section. * 

Named Ygnld consistently in SSIs. 

Corrected SSI offset "in scenario in PlacementMsgHandler " 
section. Added corresponding note in pseudo code. . 

Made CacheManagejr diagrams only show fields &mfethods * ■ 

• • used by calling handler. ' 

Added CacheMagager diagram to CM section with all fields 
and methods. 

Fixed inconsistencies in a inva^c3ateC^Rege^acMnstance , 

andpolicySaysRegenerate. " 
Component forward path changes: used component 
. palhAtPg in SSi & regen request instead of fwdpalli; , 
removed vgnComponent QS; added component regen ' 
headers.' 

=>lacementMsgHaadler handles wfiLookup even on duplicate 

placements; also sets inst parenfUriAtWs. 
Vdded dynamic-msg. 

^dded closed issues - WebServeriLookup Concurrencv and 

Redundant.metadaia-msgs. 
JorrectecJ stage^cds-dg diagram and description in intro. 
^dded conditional.transition to Regen state to Instance state 

diagram. 

teleted minheap icon from concurrency architecture diag.' 
hanged pathAtWs to urlAtWs. 
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, 12 19/27/02 David Caldwell 



KV4/02 



David Caldwell 



coBcurrent ' 

expiration occurs. • - - 
Renamed D1RJROOT to CACHEJDIR. 
>*oted ftat COMPOr®NT_SSLPREFIX will be eoawav 

CACHEJRELB^E5CTBNSION_SSI & . 

CAC^J^jsxiENSIONJ^O^SI. changed "'• 
• sarnie changes; moved to cache-level 
^erate-last^sed.interyai - reference WS plugm doc • 

/<* explain of WS last access oddities 
Added list of metadata.clients to monitoring section 

Ad SXr M i ,ToQ mdny cache mes S» a a*a°ry," 

Configuration bootstrapping," "Configuration 
teardown, , 'fflid"Fai'lover." 
CDS.levelconfig.vars are now at the stagercdsrsite or 
• stage:cds level. . 

Changed dfeld to siteld. - 

switched version to match perforce revision. ' 

^e^ 

Renamed dirRoot to cacheRoot 
Deleted dynamic-mag. . 

element before cached-lemplate element. 

Kenamed Instance parentUrlAtWebserver to 
. outerUrlAtWebserver. 

^ ^h^ 8 ^" 8 U8e il 88 «*«n request 

un for both outer templates and components. Andfbr 
components, addx-vgn-stage-id, cds-id, aite-i<T ' 
component-path-at-pg J and response-content- 
type headers. 

2££? «*■ ^dfor component?, do not 

add stage, cds, stte, and path-at-pg headerfi 
fixed processClcarCaghcMsg to handle path argument that 

than JMS. Also added Dald for TemplateSlate table 
iag into geiiexuc WwoA - added vgnagent/agW Infn 
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11/11/02 
11/20/02 



David Caldwell 



David Caldwell 
David Caldwell 



M° y * d cleaiCacheOption argument to findTemplates call ' 
mvalidateOrRegenlhstance: made inst update a single trans. 
Added CacheManager.getFuJIFilename. • 
Up&tedaearC^eMsgHanderto . 
Updated RecordUpdateMsgHandler during irrmlementatioTi. 



Switched to version-to really match perforce. version. 
Updated MetadataMsgHandler during delete & update 
implementation. •. . ■ 

Changed PlacemeniMsgHandJer to make SSI & in MD 
component offsets be the offset in the original output 
rather than the offset in the cache file. 



i 



8 



Vignette Corporation Confidential 



1 Introduction 

««» » baphto (KI vi a ^ SKK^ tenB for *« 



stage J ~ content de«ve,y server--±- 



cteployment group 



cache 
1 

1 



+ cache manager - 



web application. 



' Astageis associated vntk 1 i*™*^^^ ^ ' 

AsitehaslorinorewebappKcations a w .k > . • parage. 
ACM ' ' apP - Cabo ^ eneiates ^ single site.- 
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2 Concurrency architecture 

^^SntT * 6 ^ ^ P ° 0ls * at ^ the basic CM functions. • 
r TS™ j Pf 00 ^ fP iratI0D ^ single time-driven threads. Placement, template changes 
record changes, and clear cache messages are all handled by the HttpAction franwok thread I booT 
whose saze » deterged by the POOL.SEE configuration variabS • Xonses 

^SS?™ 8 ^^ by a Separate ftread P° o] who * size is detuned by me ' ^ ' 
REQENERATE_CONCTJRRBN'C Y JLIMTT configuration variable. * '• 



HTTP messages 



o 



Instance 
Deleter 




Message handlers that execute 
on HttpAction framework threads: 

metadata meg 

record update meg 

placement msg 

clearcache mag* 



Regenerator 



Legend 



Thread 




Thread 
Pool 



Queue 
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3 Protocols 

3,1 AM- CM m etadata-msg 

The XML schema of the body of the HTTP metadata message is in 

4,\com\vi^ette\cod\cm\metadataM8 ff .ysd. The XML document describes the addition of templates 
the deletion of templates, metadata changes, template changes, and simultaneous changes to a template 
and its metadata. MetadataMsgHandler on page 13 shows how the messages are handled. 

32 AM-CM record-update-msg 

The XML schema of the body of the HTTP record update message is in- 

ffn V:V»m\vignette\eod\cm\recordU P dateMsg.xsd . RecordUpdateMsgHandler on page' 19 shows how the 
messages are handled.- . 

3.3 PG-CM placement-msg 

r * 
The XML schema of the XML portion of the placement-msg is in 

n \^m\viRnette\cod\cm\plaoementMsg.xsd . The XML document described by the schema is 
embedded in the 1 part of an HTTP 1.1 multipart/mixed -message yfcose subsequent parts contain the 
template outputs to be cached. Eachparthas a. 'Content-length header that specifies the # of bytes in the 
part The XML document in the 1 st part describes the template .outputs in the other parts of the HTTP 
. message-and their,ijiJerreIationships. PlacementMsgHandler^n page '%\ shows how the messages are - • . 
handled. ' • ■ o ■ • 

• ■ • ' -. • . 

•314 PG-CM clear-cache->m$g " . - . ■•'«• • . • 

.The XML schema of the body of the dear cache message is in ' 

i ,V.\com\vienette\codkm\ c 1 ea rn fl r.h < .M fi rr ^ ^ XML-document described by the schema is 
embedded in either the body of an HTTP message! ClearCacheMsgHandler on page 32-shows howthe 
messages are handled. . - 
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4 Message handlers 

4.1 MetadataMsgHandler 

Handles the AM-CM metadata-msg described on page 12. 

The following diagram shows the classes that process the metadata-msg: 



MBtfldateMagHsncflar 



^^^(otefl.idrstrtoo, cdaJdStrtnn, riMd**« dUdbarife teoiyrTemptetoEnhvi 
-lwal£lateEach)nBtant»0nstBnoeaa.W5 

WT^a^^riffltrfnfl, cjtektSWng, aftftfcMio, tEnlvTcmpbtaEnlry) 

B^r^i^,^. cdattSMno. d tetdrSlrfnQ. tEfttayn-amptgtgEn^ 7 



PBriodtoBtpirqUan 



-$temp{3t8MstarJataUpdfliB(tMdtTemptet8MmadataJ 



-fra"*^ Instancaaibn ' 



^ItejpS^aseiWBtoOplw pMisSHnQ) • bdotean 
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The following pseudocode shows how the metadata-tag is handled: 

proces S MetadataMsg(httpBody : InputStream, contentLength • iritJ 
// Read the msg, then unmarshal a- 4„«-~ „ " " ' g " lflt) 
jytef] buffer -'new ££55££t£*g * Ca8t ° r MB ° bj6Ct ■ 

• intbytesRead httpBody. read (buffer) 

stririg string = new String (buffer, -utf-8") 
StrangReader str*ngReader = new StringReader(strina) 
metadata** = Metadata^g.unnarshalMeL^sgSXReaderJ 

String cdsld = metadataMsg.getCdsId 
String eiteld » metadataMsg.getSiteld 

// Begin transaction 
mdCon . setAutocoiranit ( false) 

'/* 2? or « ^^tadata-ineg if it's redundant 

.wMuiscageXd, odsld, eiteld, .Utotzy.getPathAtPg) , return 

// End transaction 
•mdCon.connnit 

. mdCon. setAutocommit ( true) 
PeriodicExpiration. templateAdd(tMd) , ' 

deieteTe np late(stageId : String, cdsld : string, siteld : string, 

// Begin transaction 
mdCon . setAutocamrnit { false) 

14 Vignette Corporation Confidential 



?£ iag J£ thatPg = tEnt ^y^etPathAtPg 

tU - •»JCon:getT e ^lat eM etadata(Btag eM/ CaeId , slteId< pathAfcpg) 

« sk Sirjssr^- if it,s redundant • . 

if tMd.getObjectrd im t^try.getTenplateMetadata.getObjectk return 
invalida t eAllInstancep { tMd) 
mdCon . dele t ©Template ( tMd) 

// End transaction 
mdC on. commit 

mdCon . setAutocommit ( true) 

PeriodicExpiration.tenplateDeleteftMd) ' ' 

^S^ 6 ^ 11 " 3 ^ 68 ^ 3 : ^lateMetadata) . ' 

Instances = mdcon. f indlnstancea rtUd) 

for each iMd in instances, ' r 

iEd-setlnstanceStatednvalid) * *' 

iMd. setExpirationTisiestainp (new Date) 

invalidateBachlnstance (instances : Li st) 
•• for each iMd in instances, ' ' 

mdCon.setTeiiflplatelnstanceStatefiMa, invalid)' 
^aate^ate.etadataistageld .string, cdsld , string , ^ ' ^ 
• ^ " ^--^---^^taS^Zd, slteXd, tEnt.y.getPatMtPg, 

if tHd-getMetadataMash ^^^SSS^^^^ retUrn ' 

^• getTea ® lateMeta data.getMeta d ataHash, return 

doMetada'taqbangeftEritry, tMd}/ 

// if any paraineters were added, 

ft ■ -> invalidate (no.regen) all instances 

// ... if any sensitive or dependent parameters were removed 

15 . • Vignette Corporation Confidential ' 



// -> just update metadata 

// ■ - 
// Notes here's an optimization that could be made to the case where sensitive or 
// dependent parameters are removed. Instead of deleting an .instance X that uses 
// removed parameter P, y_paramHasb = Z_paramHash - P. if Y_pe.ramHash' is not in 
// the instance metadata, • add it and rename X'e cache file from X j>arartHash_y. ext 
//to y_paramHash_v.ext. . 

doMetadataChange(tBntry : TeraplateEntry, tMd' : TenplateMetadata) 

// Begin transaction 
mdCon. setAutocommit (false) 

// Invalidate all instances if any parameters were added 
paramAdded *= false 

for each nParam in tEntxy.getTeiqplateMetadata.getModuleParairis; 
found = false 

for each oPararo in tMd. getModuleParameters , 
* if nParam. getName oParam. getName 

Set nParam. get Opt ions == oParamigetqptions, 
found c true 
break / 
if . J f ound, * r ± 

paramAdded = true 
break 

if l t paramAdded, . ■ 

// 'Invalidate .instances tlxat use'any sensitive or dependant parameters that • 
// were removed. 

for each oParam in ttod. getMdduleParamet ers, ' 
found = false 

for each nParam in tEntry.getTeinplateMetadata.getltoduleParaiiis, 
if nPar am. getName == oParam .' go tUame 

&& nParam. getQpt ions == oParam. getOptionsv 
found = true 
break * 

. . if I found && oParam.getType no-cache, • 

removedParamLiet . add { oParam) 
if J 'removedParamList.isEmpty, 

instances = mdCon.getlnstancesUsiiigParams {tMd/- removedPar§mList) 

. update Template metadata row (and rows in associated tables like ModuleParameter 
and TemplateState) given the values in 'tEntry 

// End transaction 
zndCon. commit 

mdCon.setAutocommit{true) 

if invalidateAll, 

invalidateAllInstances{tMd) • 
else. 5 f .J instances'. isEmpty, 

inyalidateEachlns tance ( ins tances ) 
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Periodlc^iratioa • te^lateMetadataOpdate (tMd, 
updateTemplatelstageld : string cdsid «. i . 

, tm - ^-v^u*^^, cd8ld , sitEld/ pathAtpg) 

if tMd.getTernplateHash === tHash, return 
4oTena>lateChanga(tBntry, tMd) 

aoTemplateChange(tBntry - Temn1a ^ D _ _ 

• • String pat^tpg, = S^SSSS?' = ^ateMetadata, 

// Begin transaction 
aidCon . setAutoconmd t (false J 

•• "ash = tEntry.getTernplateMetadata.getTe^lateKasn . \ . ' 

«U. setTenplateHash {tHash) 

"f 3 ' 6 » »«» Mnplatestate 
. tffPlateStates.addCtState) • ebCates 

■ Sd e t^ tT ^ lateHash,tHash ' ■ ■ 

tMd.update ^• SatTentt>lateMe tadaca.gatModCou n t) • 

// S^iSJS'iij; SlSe^otn.^ inStaac -' they've 

instances » mdCon.getAlllnstances (tMd) .' • . 

// End transaction 
jndCon . commi t* 

mflCon. setAutocommi t'< true). 
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// Ignore the metadata-msg if it's redundant 
if tMd == null, return 



it '£53222 £ ^ff^'^ — • 
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4.2 RecordUpdateMsgHandler 

Handles the AM-CMrecord-update-msg described on page 12. 

The following diagram shows the classes that process the record-apdate-rnsgi 



RecoidUpdateMsgHahdter 



*1 ) /^R9cordUpdat9J^(htt P Body:lnputStream > contertLength:lnt) 
apdate RecortStatafalaflBld^trlnn. cdsldiSMng, guttgtHnfij 



-cm 



Cached 


Manager 


r 


-InvalfdateOrRegenEacWr 


istance(!jistanc©a:Ltet) 



Regenerator 



^'tySaysRegeneratefote^^ 
-regenerate(tMd:TamplateMetadata, regenlnstancetctfrrt) 
•regenerate(tMd:Tem P lataMetadala, IMdrlnstencaMetadata) 
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ProceasRecor^teMs,^^;" contentheDgth , ^ ' ' 

// Read the msg, then unmarahal ±t 4 n v A - . _ <. 

• ruwsg .^-^dateMsg.^^ 

■ String cdsld * ruMsg.getCdald 

// Begin transaction ....... 

. .adCdn.'setAutocominit {false}.' ' ' 

. .instances = ^g^i^SSSoS^iit™?^"! °I r ' e?en ' d 

; ;xuM e g. eaUffier ^^ .. . 

. ; ^teRfecoxdStateTfstageid; cdeid; rutog.g^j ' . J;... \ - 

\ //. Bnd transaction 

indGoh . commit •* ' 

ndCon . 8etAutoccaiimit (true) 

• ^-^^^^^^^^^^ . ; 

• rcdState.setGuidfgurdj mdC ° n - fwidReco3: :<3Stat:e{9fag e id l cdtfay . „ / .. 
mdCon. update (rcdstatej *' 

• Cat ^ C ° rdDTOsNo ^i8tException e) " 

rcdstate = new Rescordstate- 

rcdState.setStageld(stageld)- ' 
rcdstate . setcdsld (cdsld) 
• rcdstate. setGuid(guid) 
ndCon. insert (rcdstate) 



Oft • 

Vignette Coloration Confidential 





dia] 




•a 


I 

1 


B 


folio 








o 


1 


5 


f 


.5 




o 




a 


e 


§ 










X 





The following diagram shows the classes that process the placement- 




■^focesspiacemen«(hl»peo<ly:lnputsiream. boundaiyStjfno) 



-cm 



-cacheRoot: String" 
-cachoFHeBctensfonSal ; String 
-cacheFilBExtsnsionNoSsi : Stifto 



CacteManager 



.■waaSfiiffitmarTvOutputStraam, oulpulOffeetlnt) 

^^tm^Outputamam, outputoteetlnt, componentPathAtPgrStrlng) 
^08eOuiputStreain(6ir8am:Ouiputsi^ m ) ■ . y B; 



path:String) ."boolean 
Instancelfcinf) 
tetedaja, IMdrl 



flegsnerator 



-feflenerateffMd.-TarfiptateMstedata, ragertnatanceli-lnl) . 
j^erate(tMd:TefnplBteMAad fl tf, | tMd:jnslancaM B > B ripfe l ) 



The following pseudo-code shows how the pkcement-msg is handled: 

tJ fattpBody points at beginning, of body ' 

proces s Placen>ent(btt P Body ; inputs tream, boundary • string 
size - getContentiefagthOfWextBart (ht^pS/taun^f ° 
read size bytes into buffer tc P soc V. Boundary) 

stringReader = new StringReader (buf f er, "UTP-8M " 

. " if msg.hasDynajaicOuterTemplate, 

. outer « msg.getDynainicOuterTen^late 
outerUrlAWs = outer . getUrlAtWs 

// Begin transaction 



// Begin transaction 

mdCon.setAutocommit (false) . 
insert = true 

. if ss^^£-~^««r ea °» «-« 

outer.g.tpathiwa) 3 ""T*- "se.getcaald, meg.getsiteld 

ins ex t » false ' 

if insert, ' 

// End transaction 
mdCon. commit 

DwJCon p a etAutocozmni t { true ) 

■ " exit, next bvte in « ^ . 

^tCantentLaaffthOCT2*PaS ( StSS ^V* 8 f ta in thfe part . 
read crlf .liintpBody : .InputStream, boundary) • i^t 

read — boundary CRLF 
^read • "Content-length- » 
read length CRLF • 
..'parse the length 
eat. the CRLF that Drscsprtt . ■ 
. return the length Ceeds ^ kody-part 

//•httpaody points at or iust nr-^r- ^ " 

Place (Plac^entMsg , >l^i^*^**»^ . 

httpfiody : inputst rwm, SLS^Sf CachedTe ^^te, . 
*/ If this ' " oute ^riAtWs : String} 

// inclu^d^n^-r in a s C t e ^ Pa ^ 0 gJ*J j^W URL at WS to save In " 

18 13t t0 ^ et the url&ws even if the 
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3 . e . 



f£ ^L^ 1 ^ 6 is ian^ed below. 

erurlAtWe.replacefO, Integer. MAXJVW/UE, cachedTplt .getUrUtWs) 

// 2SSt2"tSlSr lf CM Pr ° CeSSed 3 delete or after P G 

if tpl?Md a ^^ ff ^ etcds . ld ' PlacementMsg.getSiteld, cachedTplt. getPathAtPg) 



return 



//.metadata is bolaj updated Senerates a placenent while the template 

" SiSa'' Ktad * ta!,aSh ' = c * d »««*lt.»ietadetaH..h. 

• it srsJr s.x^xrrr.a.eSt 1 *" r 1 *-' — 15 ~— 

// is being updated. " W » Placement while the template 

I * * a 

// a placed far aa laeSSe heL^eSS • ' " 0tJler *»»«ea 

// is being updated Generates a placement' while a record 

• f s^i^^r'-"^"^"^"' . 

catch OldPlacementBXCeptibn 

if cachedTplt. hasRegenlnstanceld, 
return re9e0erate(tMd ' cac ^".getRegenlnetanceId) 

// Ignore the placement if it'* a /*,v«i4- 

// concurrent requests for tne^^Sn^ ^sT" ^ 2 PGS Pr ° CeSS 
// caches when a new WS or PQ cache ia ° ° CCUrS at Predating 

•instance maxlnst = mdCon . getIL5Si£.£ ? ffr0W systan ' capabilities. 
■ cachedTplt.get P a^^ VerBiOn0fInStanCe(tpltM<3 -^tKey, 

if maylast j. null && maxinst. state ... valid, • 
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// WS can be added after ^ D+ .« 

^•updatelnotaSK^**" • • 

// End transaction 
mdCon . cojnmit 

wJCon . setAutocoamit { true) 

.else //ccfcponent or direct p G request ' 

: if outertiriAtWs J = null' ' ' 

waxlnet . setOuterUrlAtWs (outezDriAtw B i 
*dCon .updatelnstance (maxlnstf^ • 
return r < 

ft If there's instance metadat-a ' *~ 

•version .= o. . metadata, increment its version 

if jnaxinat .1= null/ • • 

.version- maxlnst.getVersion + i* ' 

.else^t - on.cacheFile&tensionSsi' 
• «t = cm.cachemeEttensionNoSsi \ f 
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InstanceMetadata iMd = new InstanceMetadata • 
List includeMdList = iMd.-getlncludes 
for each include in oachedlplt, 

//Write the template output preceeding the include ' ■ ■ 

. while (inputldx < include. offset) 

hytetj buffer = new byte [include. offset - inputldx] 

S™: ss^Er cache fiie <-- 10 * e — ' 

inputldx +«= bytesRead 

stream, write (buffer, 0, bytesHead) 

. // S^^^'STSf??" ^ ° ffSet) —h SSI-written. 

isComponentinclude = false 

camponentPath = null _ 

Include includeMd = new Include ■ r 

included: setHandler (include, getHandler) 
includeMd, setoff set (include, offset) 
List ipList - -includeMd. getlncludeParameters 
tor each param in .include., . 

IncludeParameter ip « how XncJudeParameter " ' 

ip.setName (param. getName) 

ip.setValue(par§m.getValue) * 
' ipLiet.add(ip) m * ' ' ' \ . * .; • 

• • * * •• *.*.• 

// <include handler=»cod" of fset=°l'00"> ' 
11 <param name=» type" value=« component •/> 
"•' h Wi»3SJ '^-t-pjr value=VhostAbc/ x . js p V > 

if include. getHandler ».cod», 

if param. getName - -type". && param. get Value — -component- * 
■ isCampoiientlnclude true comppnent , 

else if parain.getName == *path-at-pg*, 
conponentPath » param . getValue 

component^ » isComponentlnclude u componentPath |. null ■ 
// Write the SSI . 

ssiLength = components ? cm.writeSsi (stream, include . of f set , componentPath, 

cm. wrateSsi (stream, include. offset) 

includeMd. setLength (ssil^ngth) 
includeMdList . add (includeMd) 
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wM?f s^£?f:2.r imt that fonows the iast c ^- fc 

•^tef] buffer = new byte [size - Wtidxi- ' 
bytesRead a httpBodv raa^h,«..T : 1 
if bytesRead ll?™' 3 *-****^™**. 6. buffer. length) 

close stream; delete cache file; w arU,- 
.throw PrematureEOF • g ttror 

iaputldx += bytesRead 

stream.write (buffer, 0, bytesRead) ' 

cm. closeoutputstream (stream) 

11 Start a metadata db transaction 
nxacon.setAutoCommit (false) 

iMd . setstate (InstanceState . VALID) 
.// Tben tha cache file i s invai^ ■ > 

-^Z^f^^ f °*° etadat * eiMseK 

// ** template deployment. 

// * * clear cache 

// . * record" deployment' 

//.Onen the cached file already -^ed,.' ^ ' 

// If the instance can be regenerated 
"^.policySaysR^^^,^^^^ , 

iL Set its state to re ^n. 
else te(InStaaC1?State - REGEH J 

• // Otherwise, delete the file & tha 
return P lacen *ntMs g .getSiteId, path) ' 
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// Write inst and ws lookup metadata 
1W • setTanplateKey ( tpltMd . getXey) 

. wSModuleValues = iMd.getModuleValues 
fofJS^ = ^-ptContentTrackinglds 

r i Silt^^ , 

Ust actionList *= iMd.getActions', r 

• aCti ? n 111 ^edTplt.getActions," ' " 

Action actionMd = new Action ' ? • 
actionMd setHandler (action. getHandler) 

.afc>Md . setValue f param . getValue) 
actionMdParinList . add (apMd)* 

else // cpnponent or direct pg request 

iMd. setOuterurlAtWs*(auteriTrlAtWs) 

mdCan. insertlJistance {iMd; 

// End transaction * 
JndCoru commit 

ifidCon . setAutocomrai t { true ) 
. /7 Send regenerate to WS 
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on- regenerate (tpltMd,- ijfd) 



// Call with autocommit false ' 

wsLookup.eetTenDlateKeyttMd get?ey) 
■wsLookup. aetS.tageld (placementMeg: getStaoeld) 

wa^o J cu P .eetPathAtPg(t M d.getPa?h?tPg' d) 
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4.4 ClearCacheMsgHandler 

Handles the PG-CM clear-cache-msg described on page 12. ' 

^following diagram shows the classes thatprocess theclear-cache-msg: 





ClegiCacheMagHandJer 












+cm\ 




. CacheManager • 1 




[ ^nvalldateQ^ReflenEachlrl8tanc9(tMd: , 


TemplateMetadata, InatansesrUat) 1 



Regenerator 

-poBc^aysReflenemteppitMdTempteteMeladata, path:SMn fl } ; boolean 
,^enefBtB(tMd:TemplateMetatteta,.lMd:lnstan ce M9tadata) 
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■Strlncrp,.^^ 9 .. J??! flta r la B (buffer, »u 



■StringReader V , ^«»»««r, "OTf-8-1 

String cdsld - ccMsg.getCdsId . 
Strang siteld = ccMsg.getSiteld 

ccMsg.getClJ^X?^,'' 6 ^' ccM S g.g etPaths . 



for each tMd i n templates, 

// Begin transaction 
mdCon.setAutocanmit (false) 



// if a match-map and a maSh-lLt W ^ CWhe API ' '■■■»» 
// in d single list since you- can't WW **** type? ' wa P them 
. // the same module name. fc have 2 outer M*P entries .with 

// tn^: b^e^c^ed^' ^ ^- instances, since • 

iLSSS rffiJS^. . CaChe -^ W111 «"« it's 

// ~ transaction -sg.ge^e,, 

mdCon. commit • 

mdCon. setAntocommit ( true) 

--WidateOr^en^chXnstanceCtlM, instancesJ 
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5 Cache Manager 



-host : String 
j -port : Int 

I -cacheRoot : String 
J -cacheFJJeExtenslonSsI : String 
-cacheFIJeExtenslonNoSsI : String 
I -wmponentSsIPreflx : String 
I -componentForwardPath : String ' 
f -modules : Map , 

-wrteSal(8tream:OutpufStr W i,oulputOfe8ttnt) 

-wrftsSsI( 6 n B am:OutpiUStream,oulputOffsethit « 

•d098OulputStream(8lream:0irtputStr9am)' 
-<tel8teFile(path:StfJng) ' - • 

-pollcySaysRegenerataftMd.TarnplateMatadata, IMrf-fnatanceM^Ho^ ' k , 
^Ik^yBRageneratBftMiT^teaMe.atlata .vSSSSSS^ 
-regenarate^Temf.la.eMetadata, raeenlns^caidS 7 " ' ; 



THe following pseudocode describes the CacheManager rnethods: 

iLf^ tla f" S confi 3 variables 
confxg. get {cacheRoot, CACirEL_RobT) 

0etfullFAleaame(8tageId, cdsld, site2d Mt)l< B 

«ua, siteld, pathAcfg, paramHasa, version, ext) ■ 
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return cacheRoot + v'fita»H 

ext . " ^ + * String. valueOf (version % 10) + «... + " 

.etoutputstreaatpath string) f 

placet would then be abandoned SSl^ 11 ?!*^ 811 eice P tion - 
need to use JNI (see section 14.7 2 I SS£? ^ J * ° Can * d0 thkt dther ' we ^ay 
different threads in ! the same process) " ^ ° pen ^ to * e » fib by 

. -iteSsifstrea* , OutputStrea*, offset , lnt) . ] lnt . 
// Build the ssi 

" Striag 8Sl ' e ° <J ~ # " + componentSeiPrefix + . w . + , ?vanId . . 

// ... ?vgnld=« + offset + -«\'-->» 

If Write the ssi as bvtes *-v,o~ 7 * 

streaxn.write (bytes j tUrn tte *' ^ tes wlttm 

return bytes, length 

wrJteSsi^treaa, I .OutputStrea*, offset^ jit „ 

£«ld the ssi ' • «*J 

. str^Jrite^esf ° Um retUr * the * bytes 'written 

return bytes. length 

closeOutputStrea^Katream : .Outputstreaa) 
deletePile (path string) 



PO^icySaysRegenerateCtMd • Temnlaf^* , • ■ 

^ Path - an.getPuliPiienaJtSd getltaSS' fif ! ^^ceMetadata) s - boolean 
tMd.getPathatPg, iMd!geWa™f^ d ' ^-^tCdald, tMd.getSiteld 
. -turn regenerator.poiicy^^ ^.getHxt2sion, 
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invali«SateOrR e gentostance(tMd t i M fl) 

• " Te ^ la teMetadata. iMd : . InstanceMetadata) • 

'/ Begin transaction 
.mdCon.setAutoccanmit (false) 

// Set state to invalid or r»«^ * * * 

"•^w^sxt r r,se,,or,te ^ 

... laate.cestete.KGB, , Ineteecestete.rawon,, ' 
iMd.update 

/ / End transaction ' 
nidCon. commit 

mdCon. s^tAutoconirait (true) 
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<> Instance deleter 

w-ai/u^ ur_uiiLA y seconds. nosc ™ has - Invalid for more than 
Since CLBA>RJPJ[>BLAY is intended to sve clients™, - 

file is deleted, we shouldn't ran into powSfSSJSf n 7°* ^ state chan S e before the 

not planning to do anything fancXffle '<22?r? w 80meoile stUJ has * W So 
occurbefom the instance metadata dde^ fft£^ ."^"eJwaioJBe.datete. defile delete will 
d ^*on*t^^ 
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7 Periodic expiration 

passed to periodic expirS soSSSf^^ 
periodically, and if it's not a^toffpS?, * 0 ?"? g fa * e tempIate if it>s « t0 
expiration for each periodically JSSSSk^i^^^^^^ 
value is a to of templates whose nextL3™ « 7 , ? he ^ 16 tftenext «P^on time. The 
"listing template's S^&^T* a is deleted or an 

^ate exotic* pr^ 

initialized. After all adWddete/uSate S ^le ^l* ^ff^' ^currently 

or another message is received Mien processed ' *• sleeps till the next expire time 

following proceSg cccuL ' M ^ 0CCm8 ' 811 * ntl 7 Amoved from the heap, and the 

for each tMd in the entry, . 
Sf^S" 7 ^^-^tValidinstances (tMd) 

cm.invalidateOrRegenBachInstance{tMd instant 
t - calculateNextExplreTimettMdr lnst ^ ce8 ) 
addBntry(t, tMd) : 
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8 Regenerator 

^POol-BachRegenerato^^^^ 

message and queue timeout ha£ SSSSdStoT P ° 01 ^ ^ e Iegenerate 

Be lowing diagram shows the class* that process regeneration: ' 




-regsnHost > string 
reganPort:1nt 



TBfleneratepMftTemplateMetacfata, instanceJd-fnfl ' 

^amtattMd.TemplalBMafadata, JMcflnetanceMatadalaj ; 

handloRegenerateMaflfmsgfleganerBteMsg) r 
lueuaTTmeoutHanclterfUmariOuifn^n^.., 



CachaManagar 



^ -componentForwardPath : String 
TTiodules ; Map . 



1./ 



: boolean 



<<lntarface>> 



fj??gg ^^^ . booleaji 



Regenerator. init 

rLJf 1 * 1 ^ 26 m confi * variables 
config.get(regeiiHoat, REGENERME KOSTl 
config.getlregeaport, REGeW^JJ " 

^^^^^^^^^^^^^^^^ 

■ . ^^^^.^tancean 
regenerate (iMd) 

" return false • «ever. 
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if tpltMd.regeneratePolicy == Always 
return true 

Date now = new Date ' 

Date lastAccess e Filelo.lastAccess (path) 

return (now^getTiae - laatAccess.getTime) / *nuBXijnu*C <= 
tpltMd.getBegenerateLastAccessedlnterval 

regenerateUtMd, iMd) 

regenerate (iMd : InstanceMetadata) 

tMd = naCon.JinclTeitplate^iMd.getTenplateKey) 
regenerate (tMd, iMd) 

regenerated tMd : TemplateMetadata, iMd s InstariceMei-Brta.-^- 

iMd. expirataonTimaBbanp + . REGENERATEL^AX w .TIME_TO_RE0UEST . 



RegenerateMsg ' 

tMd : TemplateSMetadata 
iMd ; InstanceMeitadata 



The following meihod handles the RegenerateMsg: 

handleRegenerateMagfregenMsg : RegenerateMsg)' 
tMd = regenMsg.tMd 91 
iMd = regenMeg.iMd 

rL , :!L 1DOdUleS „ add their P a «tneters to the request 
request = new VgnHttpServletRequest ' 

^?o IDOdU:LeVa i U f 9 in lMd -^™eS:ateModuleV a lue Sr 
module = cm.ioodules . get (moduleValues . getModuleNaxrie) 
if 1 inodule.iaRegenSupported, ' 
return 

" ^ e ^ le - fillReffenRe< 3 uest (reque S t ( moduleValues) 

// Sec ho3c & port 

if regenHost. length "0, 
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el J e t host & port from' iMd.getRegenHoat 
• host e regenHost 
Port = regenPort ' 

request . setServerName (host) 
request . setServerPort (port) 

« S.EoJt3gS5£S2^ th9re ' S a WS tier ' ^ p'athAtPg otherwise 

request *addHeader / «»ir ■ * 

request.ad^eader ^:^:^!S5 _P !S- at -P^' tMdlgetPathAtPg) 
••• request. addHeader »x-v™ IS 0 ?!: ' ^-SretStageld) 
• request. addHeadS /. ^ * etc ***» 

else ; 6r( X vgn - Bite -^','t24d.getSiteId) 

request . setTOl (tMd. getPathAtPg) 
if iMd.isCcoipoaent, 

// X. S3SnSS t SSE l £ B S r: ^ filter — «- Content-type of ' 
// corpoaent output to ^^L^^? 0 ^ 80 the *» CB * tr^cSTthf 
// conponent. " match the encoding of the template that called the 

request > addHeader(^vg n -respoa aa . cont e a t-t M ,e" ^ BetB 
// am n~>+ . ' ' iMd - ffetRes P o nseContent5ype) 

. "^-etContentT^ . 

// J-t and knows to .not serve the * re « uest to the pg. pg sees 

// Add identifiers of rH^-f 

^s t aV a °^ X 1 Undant "SSSuST' that ^ Ca ^ e -derations. " This helps 
request . addHeader ( "x-vm , , 

request .addHeaaer iS'S f" 1 '' <M-»etTe mE ,lateHas W 

ttue guaa ' rcd State.getGuidJ 
// Add param-hash header so , 

send request 

-■^=o^t„ SSP „^ se .^ 3 »°^ Bssi .. .. .. . • 

. // «™ aM a^ checks ,„ ^ ^ placement 
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// races conditions ong occur^nej Itl £Tl ™ **" ' 

U regeneration. For example^ Ifa tlLt^ * 6 * "^rate-policy om««. 

// already being regenerated due to a tSS? 6 ' Tf* f ° r 811 iasfcaace ttat'i 

// not pick up the new clear-cache-ij Tf £f T^*' regen 

// regen placement's olear-cache-id i* J a T? placement handler sees that a 

// wherea* an old normal ^aoSn is w'd^ "f f tS the "Ration, ' 



ei s ; f ^ 



// 
// 



// instance as-is, because a subsequS SL??' the regueBt ' Lea ™ the 

// the AM of the PG that returned Sd?L!S e ; UPdate 1,111 136 received from 
. // will succeed. So worst cS^tSrS after which "quests to that JQ ' 

iMd.setState (Invalid) 1 Mst> for each ^ ia timedOutlnstances 

iMd. update • 

8 'J Regenerate request 
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9 Configuration Space 

9 J Variables at the Wcache level 

9.1.1 J^iblic variables 
9.1.2.1 HOST • 
The name of the machine CM executes on. 

9X1.2 PORT 
Type -integer 

The port CM listens to for HTTP messages from AM and PG. 
Min-1 ' ■ 

Max-65535 
9.1.i:3 . CfCfl^JROOT 

•Type -String . ■ - ■ 

The fully qualified.path of the cache's directory root. 
9.1.1.4 / WC^DU^E^EjcrENSrON.SSI ' 

Of* - sta, CDS, Ap.Ce, m d ata, d0 projBg rf ^ ^ . 

1. Configure your web sfcrver to parse the new extension 

2. ^geCACmj^ - BXTBNSION_SS^ . 
..3. Execute VgnCDS.clearCache(new StringQ {"/*"}, nu ll) 

4. Configure your web server to not parse the old extension 
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9.1.hS CACHE_P1LE_EXTENSI0N_N0 JSSI 

Default -vgn. ' 

Dynamic change =■ start using the new value. 

9.1.1.6 COMPONENT_SSIJ>REFIX 

^^gvarwfflgoawayforoneofthefonowingreasoas;. 

For IIS web servers, the default . is "exec CGI". 

For otherweb servers, the default is "include virtual": ■ . ' 

9JA.7 P0OL_SIZE 
Type -integer 

Min - J 

Max-256 , 
Default- 10 

9.12 Hidden variables 
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we don't want to hard-code. We would KWmh^ *v 
that axe added to template m^ cZZnto Eke 0,6 confi g in V6 

not documented. But if ^c^mL^"^^ iBteM ^ seated, and they are 
necessary, but hides unnecessary comp!^^^ 8 ! UpP ° rt OT ™ to alter the behaviorif 



9.1.2.1 CLEANUP JPERIOD 
Type -integer 



The instance deleter wakes up every CLEANUP pro ion 

M . n . ^"^^JP-miOD seconds to delete invalid instances. 



. Min-0 

Max -2592000 (30 days). 
..Default -600 • 

■ 

9.1 XI . CXEANtJPJDELAY 
Type -integer 



When the instance deleter wakes uri ft rmi« a~a * « 

states have been InvaUd for more than cSSSS? •* cache file) whose 

metadata has time to propagate to all cBdSS£ sT^' deky makes sure the 



Min-0 

Max -2592000 (30 days) 
T>efanlt-300 

A2 V ^^^e S tage:CDSor S tage:CDS: S Uel e vei 

CDS. If a variable is specified for a CDS to^- !f S 48 Same for 831 056 *» " » 

• Yaltie oven-ides the CDS value! ° f Sltes associated ™* «« CDS, the CDsSte 

9.2.1 REGENESATE_HOST 
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9.2.2 KEGENERATEJPORT 
Type - integer 

The port CM sends regenerate requests * Used when REGENERATEJJOSTis not blank 
Min-1 

* » 

Max -65535 



923 KEGENERATELCONCUIU^CTJLIMIT 
Type - integer 



Min-1 
Max .- 102A 
Default -2 . 



■9X4 MGENERAITJ^AX.TIME^TO.REQUEST ' 
' Type - integer ■'".'..* '" 



Min-1 *' 



Max -3600 
Default -300 

92.S REGENERATE_TIMEOUT 
Type - integer 



The maximum number of seconds CM waits for th^ n>m>np«#i,' 

timeout occurs, the old instance is hrHSSl * ^^^nse to a regenerate request If the • 



Min-1 
Max -3600 
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Default -60- 
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10 Policy definition file 

• ZSZffi ^ nd f ^erate-policy were intentionally represented as enumerations (rather than distinct . 
boojeans like CantentChange) to simplify selection of the useful business cases. ■ UWnaJ8Unct 

10.1 expire-policy 

One of the 4 following values: 

I. ContentChange - yields fteshest pages. Template must use content tracking API. 

2 " ^S^^^^'^"^^^^^^^ 1 ' Template must use . 

" 3 ' ° 7f & S a ?? -,b d06S n0t U8e Content ***** API or uses content not managed by 

VCM) and (content changes on a regular basis). 

4 ' ^"J? ^ t /t filst f ce template is updated or cleared via clear cache API 

• ' ;™St if (content , chan Sf* «*r C ™*°I of anothei- template). The template that initiates the 
content change must use Clear cache API to expire templates that use the new content. 

10.2 expire-period 
. still heed to think about: -v , „ , 

• fflegalcdmbtoatonslu^ 

• daylight savings time changes > 
expire-period is spedfied in the fo^^^ 

and day-of-week. Bach sub-element is a string conforming to the following syntax: * 
* | elementList | everyXUnits 
*= ail legal* values 
elementlist = element, element, ... 
• elements number | inclusiveRange 
inc]u6iveRange = number-number 
everyXUnits = /number 

Days may be specified 2 ways: day of the month and day of the week. If both are specified, they axe 
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additive. 

Ibe time zone is the local time zone of the machine running cache Manager. 
Examples: 




10.2.1 minute (0-59) 

Mnutes of the hour that the template expires on 
Legal values are 0-59 
Default- 0 

. 10.2,2 hour (0-23>. • 
.Hours of the day that the template- expires on 
Legal values are 0-23 
Default -0 

10.2.3 dayrof-njonth (1-31) 

Days of the month that the template expires pn 

Legal values art 1-31 

Default-* 

10.2,4 month-of-year (1-12) 

Months of the year that the template expires on 
■ Legal values are 1-12 
Default - * 
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10^.5 day-of-week (0-6 with 0=Sunday) 
Days of the week that the template expires on 

I*gal values are 0-6 with ^Sunday 
.Default-* 

10.3 regenerate-policy 
One of the 3 following values: 

1. Never 

2. IflastAccesEedWithinJlegenmteLas^ccessedlntezval 

3. Always 

10.4regenerate-last-accessed-ihterval 

Type * number of seconds * 
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11 Plug intb generic framework 

HttsAglet and CmAglet Tie ordinal rfoXtoS , f 6 ? sets W " 'S"*^ with 2 aglets: 

• ^aament,com.vignettc:cod.cm.PlacementMsgReceiver 

• /metadata- com.vignette.codxm.MetadataMsgReceiver 

• /re <^pdate-com.vignette^ 

• ^eaiache-com.vignette.codcin.ClearCacheMsgRc 

CacheManssBrmglaOT. ^ VOT m mftated - then can get witst toy need torn the . 
ILl Message receivers 



Receiver 
Receiver 



51 Vignette Corporation Confidential 



■ ^\ 

» 

12 Monitoring attributes 

• CM 

o # normal placements 

o #regen placements ' , 
o .# instance expirations caused by clear-cache 
o # instance expirations caused by template update 
o # instance expirations caused by record update * 
.. o # instance expirations caused by periodic expiration ' 
o # failed regenerations . 

• * 

o #regeneradonsmattimeoutwaitmgtobesenttGPG 
o # regenerations that time out waiting on PG response 
• p list of active metadata clients " 

p list of disconnected metadata clients (perhaps)' 

• PG 

o time to generate . 
o # cache hits 

• WSplugin 

o # cache hits 
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13 Open issues 

2? J SfS^" Pa8€ regeneration probably never worked correctly on 

Thisis about getting cached file last access time via Win32 KndFirstRlevs ' 
GetFileMormationByHandle. The fonner can retum a stale value. What does java use? Try to '' 
reproduce the problem with COD. "wjayd.u&e, iryio 

13.2 Ticket 8233(CUST: CMD should delete directories when template is 
expired/deleted) . 

■ !2rj^^ delete, make sure to delete the 

cacheRoot/pathAtPg subdirectory afterdeleting all the instances.. 

13.3. Clearing WebseryerLookup table when WS path to TG path-mapping changes 
Do we add anew API command? : 

Do^ CMlisten for an HTTP msg sent to placement port by some standalone utility? From email from 

• fnZ^^V* We f 0 ^ m ° pti0n b 11)6 metadata 4001 to clear table or perhaps clear 
• • fl^S^MV^ ^ Pa ^ nS - ^ *» * uestion becomes ^ how does this ietXed? I" 

always E£ tS^Tl ? ^ ^ " cached P a ^ " «* CM should . 

always try to upoate the mappmg table e^ • 

f^Z^ ]S *T k the WS_PG path mapping ' sh0 "W we just add an XML element in the CDF or PDF 
to specify ws-path-pattem.topg-path-pattem mapping? . • mintne ^ or±w 

13.4 Exploded deployment and CDF external to .warjUe' deployment order problem. 
If a developer maJces changes to the CDF and a template to add sensitivity parameters andif the 
template changes are deployed before the CDF changes, there exists apoteS? *5£S 
pi tS containmg one of the new parameters will produce output CSS new 
parameter, but the instance wrl] be cached without the new parameter. So subsequent requests without 
the new parameter wxll return results that are only produced when the new paraSete^reTent 

The solution is to make sure the CDF changes are deployed before oryith the template changes. 

SF^iTr? 8 Snd a t£mp1ate * rmov < seT,sift vit y P«™*ters, -and if the " " 

CDFchanges are deployed before the template changes, there exists a potentially significant timing ' 

53 Vignette Corporation Confidential 



t 

• raj**, without the obsolete m^X^ without the obsolete parameter. So subsequent 
parameter is present. ' P ™ B ^ ra^t, fcat are ouly produeep wheu the obsolete 

tbfe 2 previous bullets are incompatible. * 

tfterefore use the same CDF), then the solutions for 

It oould wpdc hte Pe^e^ chtogehat Ctaeh^ 
7J..5 f'2b* many cache files in a directory 

problem. Jiowever, we will continue testmg tp see if it becomes a 

Configuration bootstrapping • ' . . 
Whatiaa.e^^-c^^cor,^,, We uee,^ ou. ^e ba iic aoeu^: ^ 

13.7 Configuration teardown 

^^eae^encefor'^^coBoorapoueu^ We ueed to'^ ^ basic ^ ' 

13.8 Failover . 

and cache over from the primary. • C ^^ andsI ^y catA « ^confrdofmer^ository 
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14 Closed Issues 

• i » . . * 

. fromthecache. rcurectJ y^ e ^ the component, ourplugin should not serve the request ■ 

the solution is: 

«» otfcer without ™ a *° 10 1 ** ^P«" a & ffleume a ■ : 

• ' *• . . * ' " * 

" else . . • •• ' . / 

on t. print In { "another* ); 

Do theMtowmjthinis to mate sue output is property oacta* 
• Add an entry for each URI to the CDF. 
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This also allows the direct and component instances to have different caching parameters. 
143 Ticket 57778(page regeneration doesnt do what it says on the tin,.) 
T^L^l apP T nti 7 °f he Pag * 8 " memor y- men serve from memory, our disk file's last 
wmaround.theproblem. See section 10,4 regenerate-last-accessed-interval for my attempt to do so. 
14.4 Record state guid . 

™S f 0 ** * e f 50 " 1 State §uid bein g used to ™>id caching a stale file A page 

generation using old content completes (event 1) just before new content arrives (event 2\ ™ 

A3 ^It&TT* CeV6D i 6) i nCe itS state S uid " Iess ^an the meXioris^guid 



deployment #n, rcdi 



.3" recoi-dttodatpfn, rcdl) 




4 i) 



1 


| Inst «d 




aacJbe* 




» n Valid 




a n 
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. 14.5 Clear cache ID 



. ad 








cache 


x . 




P (») Valid 




7 fx) 


9 y 




9 iy) invalid 





CM1 



3 placement (ccld^x) 



CM2 




• '11 reject 
placement (ccldnx) 
a luce jnd. 



Data 


L 2 update 


. — 2L2: 




PG2 





10 y 





inst md 




cache 




- - 1 







«I processes to SSfJ^SS ^nwo broadcasts the clear cache Oil .receives 
14.6 Instance version 

■ SSSSt' ?*■ mta- " i " *• file -4 the tastsace . 
which WS^TO^Tc^cSfe fite ^LT™ a " 0Wed fc '** *« 12 ' i» 
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l = J^ ? t 10 } A WS °£P ****** query for m instance can rctu ^ multiple rows (probably 2 at 
most).. The latest version will be used when serving the request. The version is a counter ScrSed 

tote Mb y confident the instance metadata has reached all in memory database replicas (the 
14.6.1 Regen placement overrates file 

SSS^/S 6 ?^ * ^° W i5 at Starts when ftB cache writ ten with data from content 

p^s^ 



deployment *n, rcdl 



1 recDxctUpdatefn, 




ad 




inst m£ 




cache 


n-1 
2 a 




n-l Valid 
3 Reg en. 
8 n ' Valid 




n-l 
7 a 









4 regexxfnj 



14.6.2 Normal placement overwrites file ' 
' The mismatch occuis in a window that starts when the cache file is written with data from content 

h2£^ occ^ because WS and PG serve from the cache when an 

ftf Sri JS? T f V ^° Ut regCT ' * e nKord update (event 9) would have set 

2rtkSS?e^^ r H , d W0U l d P rababJ y IWte to WS and PG well before thenew 
version of the cache file is written (event 12), so there should not.be a problem in the no regen case " 
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deployment Qn, rcdl 



6. recordgpdataln, rcdl) 






i 


•* inat nd 




cache 


n-l 

9 h' " 




? n-l .Valid 
S n * Rftffea 

13. n- Talid*; 




4 .n-l 









mSJSl ^quest is still in progress wheri the 2hdnormai placement mZ 

preemptive stake to transition.. the instance state from Regen to Valid fevent 13rt * 
placement arrive later, it's ignored, since a Valid instancfalready! ££T § 

14.7 Metadata record locking 



record 



14.7.1 Record update after CM starts writing cache file . 

i« K cas^e *" °l d pkcemMt to ^ the instance . 

•(event 7) dli^Sf^ ?* ma fir5t (event 6): 1116 Stance metadata. update 
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deployment #n, rcdl 



4 recordUQ^dateln, *cdl) 




TB 




n-l 









n-l invalid 0 
7 n-l Ragen 1 
11 n-l' Invalid 1 
11 n Valid 2 



ra o Record State table 



j cache 


8 regen(n) 


n-l abcdLO . shtml * 
5 n-l hbc6jjL . ahtml 
10 n abcdL^. ahtml 


A. 



SERIAI3ZABLB. 



ff^M wnt^bd *y the JDBC Cqnhectiqn isolation *eveU We probably want 
WonifteTittesJen developer's guide release 4.3: • *."■.: '• • 

2jj™j* SERJAL1ZABLE inaction isolation fevel, TimesTen holds all locks for fce'dmikm of the 

• Any abaction updating a row blocks out riders and writas until the transaction comiiits.' 
Any transaction reading a row blocks out writers until the transaction conpnits. 
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record update (event 6): ' 

i 

SEXISM «aqpdt. ffe tCdal d , 
mdCon. commit * - 

write instance metadata (event 7): 

" iSSEssisi- as-*-*—**-" . 

else •- . 

if Instance should be regenerated, 
instMetadata'. state =■ Regen • 

instMetadata. state = Invalid ' 

i^ktSS instanC8 metadata fields . . 1 

instMetadata. newRecord -' r 

mdCon. commit . • 

if instMetadata. state •== Regen, 

send regen to we 
else if instMetadata. state invalid 

delete cache -.file -wro-iia, _ . . 

14.7.2 Concurrent placements 

^tftheothersynchrtn^^ Thesameis ■ 

^aruung can happen wtb the.veraon, but <he solution is different. 
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I. 



.CM 



_1 placement x 
2 placement y 



FG 



PG 



Inst m6 
Xjv gtate 
3 x&y r 

5 Xiy rfra Valid 



version 



cache 
4 x&y abc^Jhsbtuil 



14.7.2.1 ^^1*«PW««^ 

Here are 2 basic options for solving the concurrent file- access problem: 

2. Use file locking to serialize file access 
. 2.. Use some other mutual exclusion mechanism to serialize file access 

147 ^ ^Weinl-concnmnUnsertQfsamerowlnto ■ 

If the concurrent file access problem is solved snthitthno™ 1 *^ * • ' i . 

thcl*thread. ThenboihUuLs i^.n ^ 9 • thread amply overrates the file written by 

.1. If the failure is "duplicate record/' just ignore it. 
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14*7.23 Adding a new state 

W 81111 60^**=^ -Mo. for 



63 ■ Vignette Corporation Confidential 



14.8 File locking 



14.8.1 V6>$ COD file handling 
• • V6 : 

o Expire without regen: rename curl.html -> curl.html.bak 
■ NT - uses a quite complicated method MTmv: 
. * ties MoveFileExW; returns if ok 

• open src file & get a file mapping; share delete, reao\& write 
•• -open dst file w/truncate, share delete; read, & write 

• if dst file not found, open w/create new, share delete, read & write 

• for each chunk of src, MapViewOfFile, WriteFile chunk to dst ' 

• close src file & mapping 

• •• open existing src file, delete on close, share delete, read, & write 

• close src • 

» Other -VU_OS -rename, 
o Expire with regen: rename temp.html k> curl Jitml . 

Calls callPageDaemon in cHents/common/generatePageImpl.cpp w/flac that WS 
writes cached pages 

• yU_OS_renarhe(tmpHle I cutfRle, removeSrc=T) 

o. NT-CopyFileW(tapHe ) curlFUe,fdIIfExists=IO: 

•wiinh^(tmpFile) 
o Other - rename 

• •■ VU_OS_unlink(curl.html.bak) • 

■• " ' . " ° MT-_wunlink ' ■ 

o Qther- unlink .*."..« 

• COD .. 

. o . Normal placement: 

■ Multiple threads can.concurrently try'to open & write the same file.' Another 
process will not have the file open, because it's a new version 

the file s.t 2 threadis blocked until stream is closed. 2 nd thread's open then 
succeeds & 2" create truncates the file •& rewrites it 
o Regen placement: 

• - Createncw version of file. Don't worry about multiple threads creating at once, 
since there will only be 1 regen placement- 
o Expire without regen: . 

- Instance .deleter thread deletes file. java.ioJ5Je.delcte fails if dst is open by 
another j vm, but it should still do since deleter doesn't delete instances till enough 
time has passed to be assured no one is still using the instance 

£gSS££2Z& *r md °* ta8MKe dsmup » tte * 
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14JS.2 Vicket 12365(1IS file locking interferes w/ PAD, CMD manipulating files) 

a proCf 0 r C o5 . ^ " 10 bU « was ^ f <*» ^eftiDy it won , be 

Original Message 

Prom: Rajkumar, Isaac 
Sent: Friday, January 18, 2002 3:12 PM 
To: Afshar, Jamshid; Caldwell, David 
• Subject: RE: AFM and COD 

-Original Message r 

From:"Raj3cumar, Isaac 

Sent: Friday, January IB, 2002 3 : 09 EM ' * ' 

To: Afshar, Jamshid; Caldwell, David 
Subject: RE; apm and COD 

This fix was in 5.0.x and lost in <; «?«.*,'*.. * 

is the first time . I a»"ear2£ Sour f ^^J' 1 ' 2 * ear3 ° f 5 -*- and. this • 

occurrence -that is not JEJ 'rSSuSSt. ' *' ^ thia is a pretfcy rara 

— ; — Original Message 

Prom:. Afshar, Jamshid * 

'Sent: Friday, January 18, 2002 3:04 PM 

To: Caldwell, - David; RajJcumar, Isaac ■' 
Subject: • RE: AFM and COD 

ResohatioTi: • . 
• Instance version, instance state, and the instance cleanup thread obviate this problem. 
14.9 WebServerLookup Concurrency 
143.1 Concurrent placement-msgs 
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14.M Concurrent template Mete metadata-nag and placement-msg 

^vSSSkllS!SSSl w IT'T ^^Wata-Macccn* 2 thread, can 

MJ3 Concurrent template add metedata-msg and placemenMnsg 

fnH j, 0 ^ 61 ? 68 8 dyDa T pfl « e at fte same a template add metadata-msg occurs 2 threads can 
14 JO Redundant metadata-msgs ? " '■' 

1. Add objectM-andmodCoimt columns to template metadata'*' . , '• ' 

■ Z ^SiS^? M -i € * tfbCa ? ^f 1 * 1 lno<IOou " t to te«npl«te addAtpdaiiaAielete nut^bbMn^ to CM On 

3. CM^etadata3V&gHand!er.addTemplate: ' 

a. if tMd exists, return • 

b. add the template md, etc 

4. CM.de3eteTerap]ate: . 

*t if iMd does not exisl, return 

b. if tMd.objectId [= mdMsg.objectld, refiim ' 

c. delete tbe'texnpiate md, etc 
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5. CM.updateTemp]ate: 

a. if tMd does not exist, return . 
' b. if tMdobjectIdl=mdMsg.objectId, return . 

c. if tMdmodCoimt >= mdMsg.rnodCount, return . ' 

d, iftMdtemplateHash^mdMsg.tempJateHa^ 
. e, update template md, etc 

before the ddetjon, wfa it conwbSr ^'.f 1 ?? 1 ff * go»4>m 

' objectM, it WMn tote. " e^^lS^i? 1 *' *** wi * * D ° dC °™. » » 
ftet^teSwarWMrtS! ^ mM ^ modC ™ ,M vedfatb e rhaad itl w a «b.ck to 1 when 

• ; app.war created with objecrld=on f modCoont^l. It contains:. .... . .- ' ' ' 

a-jsp with hash=a ; • 

b.jspwithhash=b 
This results in the following metadata: 

ajsp with objectId=ra, modCount=l, hash=a • 
. b.jsp with objectId=rn, m0 dCount=l, hash=b ' " 
app.war updated with objectId=m, modCount^. It contains: ' • 
. ajsp with hash=c 

b.jsp with hash=b - 
•This results in the following metadata: ■ - 

ajsp with objectld=m,modCount=2,hash=c- ■ »' '' 
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bjspwitbobjectId=m;modCount=l,ha8h=b - • 
Now suppose a new AM is added 

app.war is created with objectlcHn, modCounfc=2. ■ It contains: 

a. jsp with hash=c 

b. jsp with hash=b 4 

AM sends a metadataMsg containing: 

ajsp with objectId=m, modCount=2, hash=c 

b.jsp with objectldsm, modCount=2, hash=b 
If CM just looked at objectM and modCount, it would unnecessarily clear all' of b.jsp 



s instances. 
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0.1 
0.2 


1 1/10/02 
3/25/02 


David Caldwell 
David Caldwell 

• « . 


Initial version. " ~ '. : 1 

Placement protocol and placement message handler sections 
are fairly solid ' They are reviewed and feedback is 
_ incorporated Other sections, beware. 


0.3 • 

i J 

s » 


5/14/02 
1 ' 


David Caldwell 


Added instance state diagram. 
Added note to regenerate^last-accessed-interval about 
. possible WS in memory caching problem. 
Renamed physical cache section to cache manager and 
• folded the physical cache's responsibilities into the cache 
manager (since they're 1-1). . 
Added closed issues; moved 1 open issue to closed. 
Reworked placementMsgHancfler pursuant to closed issues 
Added instance deleter &-.associatedoonfig vara. 
Reworked^Iacement-msg, pkcementMsgHandler, & 
associated diagrams with new include and action ' 
1 schemes. '. 

Reject template-placements for app-server'-only if cache is ■ 
ws-onlv. '• ' 


0.4 


1 5/15/02 


David Caldwell 


Added concurrency architecture. . 
Removed placement rejection based on appServerOnly flag • 
Added .setting of expirationTimestamp, 
Rem6ved?AQ3BJGBNERATOR TYPE and 
.WBBiffiRVERJTYPB config vars. . • 


joj 


5/23/02 . 


David Caldwell ( 


Added POOL_SIZE config var. ' ~~ : — ' 

Renamed rcdDeployCount'torecordStateGuid. " ' 
Changed placemeritMsgHandler's method of detecting 

concurrent placements from "instance insert duplicate 

iow" to "file locked?*' 
Added sections: AM-CM metadata-msg, Altf-Oyl record- 

udpate-msg, MetadataMsgHandTer, and 

PeriodicBxpiiation. ' • . 


0.6 : 
1.0.7 \J 


5/31/02 1 
/31/02 r 


David Caldwell . ~ 

J 
/ 

' . * 

■ .. B 

>avid Caldwell ■ A 


\dded open issue: Clearing WebserverLookup table when 

WS path to PG path mapping changes. 

^dded cjear-cache-msg and handler sections 
^.ddeddgld. • ' 

Maintain TemplateHash in TemplateState table. 

Ldded ModulelnternalValue. 

■emoved timezone from expire-neririri 

dded RecordTjpdateMsgHandler 
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0.8 



6/10/02 



David Caldwell 



0.9 



0.10 



6/12/02 . 



6/13/02 



David Caldwell 



Added Regenerator. Changed PlacementMsgHandler to: 

• Handle regen placements • > 

• Use higher-level Module Values metadata class ' 
ReimedMetodataMsgHandler.updateTemplateMetadata. 
Added open issue: Exploded deployment and CDF external 

.to .war file deployment order problehi 
Added periodicBxpirationTimestamp processing 
Added HOST, PORT, REGENERATE HOST, & 

REGENERATEJPORT config vars. 
Moved message receivers under generic framework. 
Put some s tuff in Monitoring's ection. 



David .Caldwell 



Conform to newModuleParams metadata-msg structure: 
I Handle placement-msg in 1st part of hop msg rather than 
preamble + subsequent part lengths are now Content- 
length headers in eac h part. 



0-11 7/11/02 I David Caldwell 



Minor formatting changes. 
Increased CLEANUPJ3ELAY to 5 minutes. 



Corrected "PG Placement message to CM"~diagram in' 
section 4.3. 

Added references section.' 
Named vgnld conslstently.in SSls. 
Corrected SSI offset in scenario in PlacementMsgHandler ' 

section. Added corresponding note in -pseudo code. • 
Made CacheManager diagrams only show fields & methods 
• • used by calling handler. ' 
Added CacheManager diagram to CM section with all fields 
and methods. 

Fixed inconsistencies in a invaBdateCrRegehBachlnstance 
andpolicySaysRegenerate. ' 

Component forward path changes: used component 
. patbAtPg in SSi & regen request instead of fwd path; - 
removed vgnComponent QS; added component regen ' 
headers-.' ' ■ 

PlaceinentMsgHandler handles wsLoolcup even on duplicate 
. placements; also sets instparenfUxIAtWs. 
Added dynamic-msg. 

Added closed issues - WebScrverLooicup Concurrency, and 

Redundant.metadata-msgs. 
•Corrected stage^cds-dg diagram and description in intro. ' 
Added conditional.transition to Regen state to Instance state 

diagram. 

Deleted minheap icon from concurrency architecture diag.' 
Changed pathAtWs to urlAtWs. 
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9/27/02 David Caldwell 



10/4/02 I David Caldwell 



deletes cache file when"^^ 
expiration occurs. • • 

Renamed DULROOT to CACHE_DHL 

CACHE^iLE_EXlENSION_NO_SSI.changed- 
. dynamic changes; moved to cache-level 

"BS i^ c ^- iDt ^ai - reference WS plugb doc • 
foz -explain of WS last access oddities. 

a£K ^ ° f met3data ^ to m °nltoring section. 
Ad^ed open 1SS ues: "Too many cache files in a directory," 
I ConfigoratiOD bootstrapping," "Configuration 
teardown," and Taflover." 

CDS-level config.vars are now at the stagercdsrsite or •' 
• stagexds leyel. . OT 

Changed dfeld to siteld. . * 

pitched version to match perforce revision. ' 
^eS S5. fr ° m ^ *~ «*» to meir 

Renamed dirRoot to cacheRoot 
Deleted dynamic-msg. . 

element befwe cached-template elemenL 

Renamed Instance parentUrlAtWebserver to 
. outerUrlAtWebserver. " 

^ S^ el ! SerVeriS ***** useit asregen request 
C ZS b0t ? 0U ^ tem P late s and components. AndfS 

• co ^POnents,addx.vgn-stage-id,cd S -io; site-id, ■ 
component-path-at-pg.and response-content- 

. type headers. 

^T^' 0 '- ^ for components, do L 
Pfw g ' ™ * ^ and P at h-at-pg headerS 
fixed processClcarCachcMsg to handle path argument that 



I p,™f' AJs r ° "*W Dald for TemplateState table. 
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14 • 


1L7/02 • 


David Caldwell 


Moved cleaiCacheOption argument to findTemplates call ' 1 
invalidateOrRegenListance: made inst update a single trans 
Added CacheManager.getFullFilename. • 
Updated ClearCacheMsgHandler during implementation 
Updated RecordUpdateMsgHandW during implementation. 

Switched to version to really match perforce version 

Updated MetadataMsgHandler during delete & update 
implementation. 


~L6~ 


' 11/11/02 


David Caldwell 


17 


11/20/02 


David Caldwell . ■ 


Changed PlaceroentMsgHandJer to make SSI & in MD 
component offsets be the offset in the original output 
rather than the offset in the cache file. i 
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I Introduction 




— content delivery server-±- 



dsployment group 



cache 
1 



cache manager - 



site'-- 



•+ 



web application. 



. . "^•^^^S^(a^ ) .ACDSi.i J1 onl yls i as! 

A cache contains. ' . 

Jployd to Application M^^fflSS ^ c ^»n Hlc (CDF) tha, ia ' 

CM saves the tempiate-Jevelcac'hP /i^ • J . 
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•inandPG- 



• template-level and tatanca-level metadata are treated to read-only US e b v * TO phg. 

St^t'm^oe^^^^ 

P as upoate, periodic ejpiraticm, content change, or tie dear cache API command ' 

reooest par=o^SferKuea?u I af^f ra S° n "T" Wfll - COn0in fc «"*«<»si<ive' 
I J Instance state diagram 

CM clients will only serve from .the cache when the stale is Valid or Regen. ■ . . • 



gaaimwtjl earmaaA «plmj 




lb. -plac™* [comnraren, wcpiraj- mot oolnrs when 1 f 
proc. M „ • tanplBt* or itonM^l^t or J elm «*.. 
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2 Concurrency architecture 



' St^S e "°^ «p 1 rat I o D are single timeHdriven ti^ads. Placement, template changes 
SifSS^^ ^ haDdled by 1136 ***** fr^oric thread 



HTTP messages 



o 



Instance 
Deleter 




Message handlers that execute 
on HttpAdbn framework threads: 

metadata' meg • 
■ record update meg 
placement meg 
clearcache meg* 



Regenerator 



Legend 




Thread 




Thread 
Pool 



Queue 



- in 
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3 Protocols 



3.1 AM- CM m etadata-msg 

The XML schema of the body of the HTTP metadata message is in 

..V.\com\vi^ette\cc)aV^\metadataM a p ™d. The XML document describes the addition of templates, 
the deleton of templates, metadata changes, template changes, and simultaneous changes to a template 
and its metadata. MetadataMsgHandler on page 13 shows how the messages are handled. 

3.2 AM-CM record-update-msg 

The XML schema of the body of the HTTP record update message is in- 

„ 1 V:\coi q \vignette^cod\cm\recordTJpdateMgg.xsd . RecordUpdateMsgHandler on page"l9 shows how the 
•messages are handled.- • 

3.3 PG-CM placement-msg 

The XML schema of the XML portion of the placement-msg is in 

rl l.ko^\vi^ette\codkiq\ P IacementM8 ffTfsd. The XML document described by the schema is 
embedded m the 1 part of an HTTP 1.1 multipart/mixed message whose subsequent parts contain the 
template caputs to be cached. Each part has a iContei^ength header that specifies the % of bytes in the 
part. The XML document in the 1 part describes *he template.outpvtts in the other parts of the HTTP 
. r^ssage-and the 1T ,wtenel^ 

314 PG-CM clear-cache-m$g • ;•: « ■" ; 

.The XML schema of the body of the clear sache message iain " 

A.Wvi^e^ xsd. .The XML- document described by the schema is 

embedded in dtbtf the body of an HTTP message! ClearCacheMsgHandkr on page 32- shows hovr the 
messages are handled. * ^ 
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4 Message handlers 

4.1 MetadataMsgHandler 

Handles the AM-CM metadata-mag described on page 12. 

The flowing diagram .hows the classes that process the metadata-msg: 

>1 



MaladSlaMsaHBndlsr 



■^aaaMjdatBM.olwipaafyrtoputSlmam, contertLanfltMnt) : 

^fTemptewt^oattsirtne, cdshiSttno. at B ld:SMn B . 'WASWnir.ta.tvTBmsletoEnM 

^MitaAlHniitaniasOMd/TBTnpIalaMBtadaUO 
-JnvaldtoE8ChJnBfaf^(jnBtanQ8aA.la^ 



♦cm 



' CaeJn 






-invanilataC>flop0nEachJiistEmoo(IM 


J^^IaJaMetadaia t Instanceaiiso 



-$templaieM8lattotaUpdale(tM±TampteMtodat^ 
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Ita foflowmgpseudo^odeshows how the metadata-tog is handled: ' 

Pr °/^d a Sf^ fh S PBOdy 1 ^"tream, contentLength' : int) ' 
// Read the msg, then unmarshal it into « rao *-^ LZ ' . ; 

yyt-N buffer « new byte [contentieagSj ^ MB C * 0 "* • 

/ int bytesRead «" httpBody. read (buffer) 

String string = new String (buffer, ■otf-8-) ' ' ' 

String stageld = aietadataMsg. getstageld 
String cdsld = aietadataMsg. getCdeld ■ 
String siteld « aietadataMsg. getsiteld 

else if *«J? ZZ2Z f S f- ' cdsId ' siteld, textplateEntry) - 
else if templateEntry . getTeroplateBvent update-w* 
% updateBothfstageld,. cdsld, siteld, teajSLt£T' 

aSdTeznplate (stageld : String, cdsld • Jh^«„ ' tu « 

t // Begin transaction 
mdCon.setAutocommit (false) 

ii S^^^tafcta^g if it's redundant 

if radCon.tenrplateBxiatsfstaCTerH '^tj iv „ • • 

stageld, odsld, siteld, .tEntry.getPathAtPg) , return 

insert Template metadata row tMd tan* * *, 

// End transaction 
-mdCbn.coinmit 

. mdCon. setAutocommit ( true) 

PeriodicExpiration.teipplateAdd(tMd)." . 

*t^W'«S&SI ' sai *°- • lteM 1 st ^-- ■ ■ ■ 

// Begin transaction 
mdCan. setAutocommit \ false) 



14 Vignette Corporation Confidential 



t™* 3 * ^ thAtPg = tEn try:getPathAtPg 

tm . mdCon :ge tT en5) lat eMe tadata( 6 ta S e M/ cdald, siteld, pathAtPg) 

it sk Sirssr^- if ±t,s * • . 

if tMd.gatObjectrd tEnt.y.getT^lateMetadata.getObjectld, return 'i 
lavalidateAllliistances { tMd) 

indCon.deleteTeinplateftMd} • 

delete WebServerLookup rows with template's starred , _ " 

utsn^Aate a stageld, cdsld, siteld, and pathAtPg 

// End transaction 
aidCon. commit 

2ndCon.setAutocommit(true) ' 

Perio<±LcE^piration.templataDelete{tMd) *" 

iavalidateAlllnstancesftMd : Tes&lateMetadata) ' 

for each iMd in inatances, ' r 

iMd.setlnstaaceState (invalid) * ' ' 

iHd . setEaqpirationTimestaap (new Date) 

invali^tGBachlnatancefinatances : LiBt) 
•• for each iMd in inatancea, ' ' ■ 

mdCon.aetTemplatelnatanceStatefiMd, Invalid) 1 

^te^late M etadata,sta g eXd ^string, "cdsld /string, ' Blfce id ': string " ' •• " 

tMd * -dCon.get^laSada^^^Id „ ■ " 

wca (stageld, cdsld, aiteld, tEntry.getPathAtPg) 

* x z°:: 'SSiTSSSr* lf it,s ^ dant " • ' ' • ' 

if tMd.getMetadataHash « SnrS ™ ate * Ie ^data. getModCount, return • 

tEntry - getTeffl f 1 ateMefadata.getlletadataH a sh, return 
doMetadataghangeftEntry, tMd) / 

// if any parametara were added, 

// • -» invalidate (no .regen) all inatancea 

15 . • Vignette Corporation Confidential ' 



// -> just update metadata 

II . 

II Note: here's an optimization that could be made to the case where sensitive or 
// dependent parameters are removed. Instead of deleting an instance X that uses 
// removed parameter .P, • Y_j?aramHash = SjparamHash - P. If Y_paramHash is not in 
// the instance metadata, • add it end rename X'e cache file from X_^aramflash__v.ext 

doMetadataChangettBntiy : Tempi at ©Entry, tMci : TemplateMetadata) 

// Begin transaction 
mdCon.eetAutocommit (false) 

*'•«•■. 

// Invalidate all instances if any parameters were added. 
paramAdded « false 

for each nParam in tEntxy.getTea^lateMetadata.getModuleParaais; 

f o\ind = falae ■ 

for each oParam in Ufld.getModuleParameters, 
* if nParanu getName p= oPar am. getName 

' . ScEc nParam.getOptions — oParam.'.getOptions, 
found » true 
break 

if . i found, • r * 

paramAdded = true 
break 

if I, paramAdded, . . 

// 'Invalidate -.instances that use'any sensitive or dependent parameters that'- 
// were removed. • 
for each oParam in tM. getMdduleParamet ers, ' 
found = false 

for each nParam in tEntry.getTe3r©lateMetadata.getModuleParams # 
. if nPar am. getName == oParam .'getName 

&& nParam.getQptions oParam.getOptions-, 
found = true 
break ■ 

■ . if 1 found && oParam.getType 1 = no-cache, • • 

remov.edParamList .add(oParam) ■ 
if rremovedParamList.isEmpty, 4 

instances = mdCon. get rnstancesUeiiigParams [tMd/ removedParamList) 

. update Template metadata row (and rows in associated tables like ModuleParameter 
and TemplateState) given* the values in 'tEntry 

// End transaction 
mdCon. commit 

mdCon . setAutocommit (tame) • 

if invalidateAll, ' 

invalidateAllInstances [tMd) • . " 

else, if J instances'. isEmpty, 

inyalidateEachlns tance ( ins t ances ) 
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^riodlcWiratic.teznplateMetadata^datef^, 
update T e^late( e t ag eId : string, cdsld : String siteld 

if tMd.getTemplateHash == tHash, return' 
doTenplateChangeftSntry, tMdj 

// Begin transaction 

mdCon.setAutoconodt (false) • 

•• tHash = ^try.get^lateMetadata.getTe^lateflash . \ ' . 

t^.setTenplateHash (tHash) 

• " tate " new TtoplateState . '.*' ' 

• tenjplatestatea.addftatate) cestatea 

• 25 a * e :fJ"^PlateHash(tkash) ' • . ■ • 

tMd.update ^ getTentt>lateMe tadata. ffe cModCouat) ■ 

// Get this- template's vallrt ' 

// already bee/ been cleSed S ^e^ ^ ^ dances, since they've 

instances » mdCon.getAllmstances (tMd) .' . . 

// End transaction' 
flidCon. commit 
- ndCon. setAutocoznmi t'{ true) 



: ^WidateOrKegen E achXnstance( tM d ( instances) 



updateBothfstageld : String, cdsld . c* ; 

^ - tEntry, T^lTte^t ' * tZing ' SlteM • "ring, 
tMd = wSCon.getTemplate^etadatafsta^M 

aatafstageld, cdsld, siteld, tEntry. getPathAtPg, 
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// Ignore the metadata-meg if it's redundant 
if tMd == null, return 



if 52:22352 a sssrr"^. ™™ ■ 
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4.2 RecordVpdateMsgBandLer 
Handles the AM-CMrecord-update-ms^ described on page 12. 
The mowing diagram shows the classes that process the record-update-: 



RecordUpdateMsgHandter 



4 i3 ^ S RecordUpdateJ^ fl (httpBody:lnputStr9ani,cont9ntLength:lnt) 
.-UfidateRecofdstatafataaeld^trlnp. tthld iSMnq. outtSBno) 



-cm 



CacheA 


Manager 




-InvaildateOrReganEachlr 


istancepjistancealrst) 



Regenerator 



^I^SaysReg S nerate(tpltMd:Te m plateM e ta(teta,pelh:S"wng) • boolean 
-fBgensrateftMdrTejnplateMeladata, regen Instance^ 

-regenerateftMd:TemplatsMetedata,lMd:l ns tanc e M B tari fl » a ^ 
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The Mowing p^ode aow, how the ^crd-upfc*-^ i» tadled: 

^•^co^^ 5(MWBody ;^ escr<!m . oonteotleDBth _ ^ 

// Read the mag, then unmarehal ±t „• = 
: ^ten buffer * new Jwrc^taLSw ' ' CMt0r ° bject " . 

7^ rlDg = ^ew String (buffer, -utf-8'M 

String cdsld = ruMsg.getCdsId 

U Begin transaction 
. .mdCdn/setAutoc'onffliit (false)." ' ' • • • 

// the new data. £ S', aSea^ SETk" 111 Wa * ting tD ffo to ^ttey'll . 
// record-state-guid will con^oi fi^? | h f "Sen, the placement * * 

• .Ranees = m^.g^j^SS^iS^',^'' ?^ C6 f ° r r ' egen ' d 

; ; *^^enunierateReco>dId) *c«tantl d( stageld, c.dsXdy r • ...^ ..... 

.• «pWecordState-( a tageid; c^Id; rntog. get^d) ' • ' ' . \ - *. 

. // -End transaction . 
mdeoh. commit •* '•'".*. ' 

n^Ccm.setAutoccHmit{true) ' 

• rcdState. setGuidfgu^dj IndCon -^ ndR900i :«astate( S t a geId, cdsjdr • . / 
mdCon. update (rcdState) ' .' " ; 

rcdState = new Recordstate- 
rcdState.aetstageld(stageld) 

rcdState. petCdsId (cdsld) . • 

• rcdState. setGuid(guid) 
mdCon. insert (rcdState) 
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I 



jr 




• 1 



The foDowing diagram shows the classes that process the placement- 



•msg: 




■ff)foc8ssPlacemeni(httpBody:InpufSlream, bounda^Sirtna) 

— ^ r^ nuviti ^ ri ^ mm\MBQ, cacnBcfTpfcCechedTemplate) 



-cm 



-cacheRoot: String* 
-cacheFilBExfensfonSal : String 
-cachgHlBBctBnsionNoSsI : string 



CacheManager 



.■wifaSsifstreanvOutpulStmam, outpuIOflaetlrt) 

■cloaeOuiputstreflm(6lream:OiitputStream) 1 . W • 

■<teletaR)e(path:Strfng) ; • ' . \ 

■policySayBRepenerataPMdTampteieMetadalB, path:Strino) -boolean 
^8fateflMrtTen^ateM B tadata,ln6lanceld:!m) ■• ■ - 
-reg8nemt 8 (lMd:TeniplateMBtadata.lMd:i nS tene a M BlB rf nta < 



floflenerator 



-pollcySaysRagenarataftpltMdrTcmplateMetadata, palh;SWn 8 ) : boolean 
-reaenerataOMd.-TertplateMetadata.resenlnstancalddhfl . " 
_ . [,TB B anerate(tMd,TemplateMated fl w rMdilnslancatotorfatai 

The following pseudo-code shows how the placement-msg is handled: 

U fattpBody poiats at beginning, of body ' 

processPlacemenMhttpBody : InputStxeam, boundary • strina) 
Sad s^T^TM >~ 

. - if msg.hasDyQamicOuterTemplate, 

outer = msg . gettD^ainlcOuterTeinplate 
outerUrlAtWs = outer . getUrlAtWs 

// Begin transaction 



outeroriAtWs * outer, getUrUtWs 

// Begin transaction 
m<2Con .aetAutocoOTiit (false) 

insert = true 



. " S5S^«5=3^^ °» — — ^rt 

// EVES"* ^ 'th. t^ate.to cacheable " 

. . if mdCon -taBTapplateitetadatadnag getSte^M 

outer. getPathAtPg) 0 9 etsta ^Id,. msg.getCdsId, msg.getSifeld 

insert e false • BJ,a ' 



false 

if insert, 



wsLoo^.set^^^^tUrlAtWs) 
w^ooJtup.setStag e id {l nsg getstacrel^ 



// End transaction 
mdCon.coiranit 

n4Con. aetAutoccanmi t { true) 

. ^late, httpBody,. boundary, euterUrlAtWs) 

• /y .°P° n exit, next byte in ht-tns^, ^ 

gatContentLsngthOfN^Par? (JtSSv data ln «* « part . 

readCSLF pBody , .Inputstream, boiaodary) " Lit 

read — boundary CRLP 
t read • "Content-length: » 
read length CBLP • 
. .'parse tbe length 
eat. the CRLF that orecspfle v ^ 
. .return the length P " Ceeds ^ ^dy-part 

//•httpBody points at or just D rinr - • " 

Place (placementMsg ; Pia C ^Pn + S 9 t0 ne5ct boundary- 

*/" If thi i h ' ou t e rgrlAt Ws : String} 

// indued COnpoVS^ in% C t h6 ^ 0 £~ g- W ^ at WS to 8ave ia 

■ 18 39t t0 9 et the url@ws even if the 
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. iv, integer. MMUTOLUE, cachedTplt .getUrlAtWs) 
// J2St3"£lSr " ^ Pr ° CeSSed a d6lete w W le or after PG 

if tpltMd == ' ' placement *teg.getSiteJd, cachedTplt. getPathAtPg) 

return 



//. metadata is being updated Jeneratea a placement while the template 

// is being updated. " M geaerates . a Placement while the template 

' // js.'irs.'is r?r^L a cl «f a£t ~ » ■ 

II a placement for an inet,^. ^7"!?"!" " ear another generates 

« f*»^ ^O^ffS^'^ 1 *-* PG generated 

// is being updated. generates a placement while a record 

" "atarTSTreT^raSon"" 00 — «- -.generate, 
catch OldPlacementExceptibn 

if eachedTplt.hasRegemnstanceid, 

return re9enerate(tMd ' Cache ^"-^t R egenI M tanceId) 
// Ignore the placement if it's a du n 

// concurrent requests for the same instance ^ e 5 ccurs wh ^ 2 process 
// caches when a new WS or PQ cache is S ° ° CCUrs at P"«iatiag 

instance maxlnst = mdCon.get^VerJi^nf? ? t0 0IW ,VBtenr capabilities. 
• cachedTplt.getPara^^^^^^^^^^^.getKey, 

if ma^clnst 1- null && maxlnst.state ... valid, • 
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// WS can be added aftpr 

^n. U pdat e i M tJ«SSS, ed,91t * at ^™M • • 

// End transaction 
ffldCoa . copmit 

ffldCon . setAutocommi t ( true J 
. else // cdnffoaent or direct PG xeguest ' 
* if outertirUtWs J « null' ' 

return r * . 

// If there's instance metadata 

•version = o. . metadata, increment its version 

if jnaxinst j» null/ 

^.ect » cn.cachepiieactensionssi' 

ext = cm.cachePileSrtensionNoSsi ' * ' ' 

// Build the cache -f il en = mtt ' -u 

cachedTplt.getParamHash, version; ex?) ' eaoh ** lt -SetPathatPg, 

// only^iiw^^ff™ Can " fie ^.nio'.channels Filed,™ 

^^tt^S^f Placen.entKsg.cdsXd, 

catch 5iiSi^^ 7 £^ i si issr - ^ - • 

. return . 

inputidx = o s " 
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InstanceMetadata iMd = new InstanceMetadata : ' • 
Last includeMdList = iMd.getlncludes 
for each include in cachedTplt, 

/v.^ 1 /® the tem P late output preceeding tie include ' • ■ 

. while (anputldx < include. offset) "elude 

Sflii = nSW ^' te Include. offset - inputldx] 

SS SSrST' cache me? - 1 * ' • 

inputldx +o bytesRead 

stream. write (buffer, 0, bytesRead) 

. // St^L^lala'ir^ f 6 ° UtPUt ° ffSet) ^-chSSl-written. 
// directly to rhf^^ t ?\ iastance metadata so the filter can skip ' 

isConrponentlnclude = false • • 

camponentPath o null . ' 

Include includeMd = new Include ■ r 

included; setflandler (include, getHandler) \ 

includeMd. setoff set (include. off set) ' 

• List ipList = includeMd. getlncludeParameters 
tor each par am in .include f • 

S e i2S ar T iter ±P = haW Inc ;^?arameter ' ' 
ap.setKame(param.getName) 

ip.setValue(param.getValue) ' ... 

■ ipl.ist.add(ip) '.».•' • 

// <incliide handler=''cod» offset="l"00»> 

'I <param name=«typ e » value=» component '/> 

/! WinSX —-"^t^ value^/hostAbc/x.jsp-^ 

If include . getHandler _ == »cod-, 

if paraiiugetName ^type". k& Daram rrpH7 3 i„« 

else if param.gettfame == *path-at-pg*, 

conjponentPath = paranugetValue ' • " ' 

componentSai isCoxnponentlnclude componentPath i= null ■ ' • ■ 

II Write the SSI . 

cm.wratessi (stream, include. offset) . 

inclxadeMd. aetLeng'th (ssil^ngth) 
includeMdLi a t . add ( includeMd ) 
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. ^fLSff^r^ that f ° ll0WS the last ^on-t " 

^SsLd t?*f y - read ^«-, 0, buffer. length) 

close stream; delete cache *n». n 
.throw PrematureEOP Caclie . file ' log error 
inputldx += bytesRead 

stream.write (buffer, 0, bytesRead) ' . 
cm.c2oseOutputStream(stream) 

^rf tart J metadata ^ transaction 
sxaCon.setAutoCozmnit (false) . \ ■ 

iMd. setState (InstanceState. VftLID) 

ratm= » laoe «»»ateff-setSit e ia. path) ' 

// *• template deployment 

// ' * clear cache 

// • * record* deployment' 

• ^^^^ • 

U. Then the cached file already • expired..' 

// If the instance can be regenerated 
if «-P0licySe VS R eee nera te( t2JS? t ^i l)/ 



4 «/ 6t ltS State to re 9en. 

iMd. setState (Inetancestate .REGEN) 
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// Write inst and ws lookup metadata 
i«d . setTemplateKey ( tpl tMd . getKey) 

fof -S^-i = ^-^tContentTrackingXcls 
List actionList & iMd.gettctiena '. r 

f or each action in cachedl-plt.getActions ' * " 

Action actionMd - new Action ' . '• 

aSd 0 ^ 8 *^ apMd = aew ActionPar^eter • • •' ' 

. apMd.setNamefparam.getName) 

.at)Md . setValue (param . getValue) 

actionMdParmnList.aad(apMd)" 

else // component or direct PG request 

iMd. satOuterUtlAtWs'CouterTJrlAtWs) 

jndCon.insertlnstance(iMci) ' ' • 

// Bnd transaction ' 
mdCon. commit * 
mfiCon . setAutQcommit ( true ) 

//If another . thread Drnrpoc^ «~ . ' 
. / / Send regenerate to WS 

30 Vignette Coiporation Confidential 



cm . regenerate (tpltMd, • ijffd) 



// Call with autocommit false ' 

WebServerLookup waLoofcup = new Webserver n „k„ 

^oo^p.setTe» ff lateKey(tMd Jet^r 
•wsLooJcup. setS.tageld (placeraentMsg; getStaoeXdl 

•"^^p.eetCdsIdfplaceineaWtegJefcdfidr ■ 
vreLootaip. BetSiteldfplacementMsg getSiteld) 

v^oJcup.aetPa^tPg^d.getPa^p^ 6 ^ 

■mdCon.lnsertWebServerLooJaipfwaLookup • ... 
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4.4 ClearCacheMsgHandler 

HMdlesthePG^clear^ache-msgdescribedonpageU * 
^following diagram shows Che classes thatproqes* the clear-cache-msg: 



ClesfCacheMagHandter 



^ ro ce Ba Cl9 a ^acheM 3g (ht1pBQdy:lnp U BSt re am,CQnt a ntL fl nnth- l nt ! 



. Cacheh 


Manager - ~| 




•fnvalidateOrFtegenEachlnatance(tMd: 


TampfateMetadata, fnstancesrLisfl 



Regenerator 



-poircySaysRegsnemteOpUMATemplateMGladate, path:Strtn fl ) : poolean 
-xegenerateOMcCT^ateMatadata, reganlns^ceLt) . ' 
-gnefBtettMdiTemplateMetadata.lMd^stanceMflfBrfat^ 
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8halClearCaclleM ^(stringHe a d e r) . 
. String siteld « ccteg.g etS it e id 

ccM 3B . getclearC ^ t a ^(-tageid, cdsld, siteld, ccMsg.getPaths, ' 
for each tMd in templates, 

// Begin transaction * r 

mdCon.setautocommit (false) * ' 

* 1 

. // the same module name. fc have 2 outer Map entries yith 

If Get the valid instances t« , 

// f r' V \ alrea ^ ^S^ e JLS?S d - ^ rS5en stances, since 

// *nd transaction ' " ^'^^ Q.tMatfcid.t) 

mdCon, commit • 
odCon.setAutocoamit(true) . 
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5 Cache Manager 

The following diagram shows the CacbeManager class data members 



■host : String 
-port : Int 

-cacheRoot ; String 
-cacheR/BExlanstonSsJ : String 
-cacheFUeExtenabnNoSsI : String 
-componentSsfPreflx : String 
-componentForwardPath : String ' 
•modules : Map 



CacbeManager 



•wrileS6J(8t re am.-OutputStr § am,outputOfeet-lnt) . ' 

HtelsteRetpatfrStrlng) 1 - . 



The foflowmgpseudo^ode describes the. CacbeManager methods: 
// Inxtialxze my config variables 

coafig.getfcach^ot, aonJEb? • • 
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return cacheRoot +«/"'*• 
. ♦ PathAtPg + : ££™£ V; : cdsld + ♦ siteld + v . 

ext • . + " + ' "ring. valueOf (version % 10) + ...» t " 

setOutputStrea.fpath string) , '^^^ 

• ? t0 ^ 

placement would then be abandoned S t v ^ M « Option, lie 
need to use INI M ? j ^fi^ 10 ca * * d °^t either, then we may 

different threads hi the same process) " open and write to the same file by 

. vritesei (stream : OutputStrea*, offset : lnt , \ lat . 

// Build the ssi 

String ssi'e *<i , „ 

„ ce • • ^onen tSsi p refix , « w . + + ^ + ^ _^ 

^•^eiV'LfSi^^g^vtat utf-8 won't-hurt, 

// Write the esi as bytes ' * • 

return bytes-, length 

tfritessi (atrean, : .OutputStream , offset 1 ii " tr 

* ' C ^°^~ g . : Strlaff) , iiat 

String ssi * *< i _ 4 * • 

// Write the ssi as bytes the*,' ~ - ' 
.stream.write(bytes) ' en retUr * the • bytes written 

return bytes. length 

doseoutputstreaxn^^ : .OutputStream) 

•SeleteFile (path : string)' 



^ ^^^^^ SS " £f*^taW : - boolean 

tMd.getPathAtPg, iMa g et Pa™5 C , t H 4 ' WtCiJald, tMd.getSi te Tr3 
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^^fis:'- ■ ^ lateMetadata , ^ es t . . 

invalitoteOrRegentostanceftMd, Iwa, 

• invalidateOrR4genlnstar>ceftv^ m 

// Begin t ' —"«-«-«.. IX ■ . 

U Began transaction 
mdCon . setAutocommit (false) 

/•/ Set state to invalid or MMm r - 



iMd. update 

/ / End transaction 
mdCon. commit 

mdCon. setAutocommit (true) 



^validateOrRe ffenEaclUnstaace " V ' • 

for each iMd in 'instances S ! Llst) 

tm = mdCon.findTemplateaMd) inva1 ^ „ 

' ' ^^^OrRegenlnstanceftlid, i*d) 
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6 Instance deleter 

CUSANUPJDEUY Mccm*. file) whose state ^ ban Ihvalid for mm am 

n» no! placing to do mythiog U^StSEE^n- -° ^ someme stai *» * 9P«. So 
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7 Periodic expiration 

passed to periodic expiration so it « SSffiSf metadata , chan g^> & template metadata is 
periodically, and if it's not already iSfi P S?? S for fte tem P Iate if « to expire 

expiration for each periodically aiSS J?"^?^^ a "*» hea P of next- 
value is a list of templates ^S^JS^J^ ^1° ?* hea * is *° next «]*«*» time. The 
existing template's jLJ^?^ tT^* 6 *^" Wheo fl ^P Iate is dieted oran 

terminate expiration proceaSSSi 2^ Js ^ to ^on so it can 

initialized After all idSate SJS^ 8 "* "? f pire P eriodi ^v, and it's currently 
or another message is receive! 3 e Skeps 131 fte ^ 

followingprocessing occurs: P Reoccurs, an entry is removed from the heap, and the 

for each tMd in the entry, . 
S^SLii?^? ^alidlnatances (tMd) 

addEntryft, tMd) ... 
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8 Regenerator 

■x-V aid qoet* toecm I-AJSSSSSST f "*" ^ Kge " OTte 

» stows toe chssB that process regciera ti 0]1 . 



| -reganHost : String 
*ort : tnt 



Regenerator 



-resenoratepMATemplateMetadata, InatencaJdlnt) ' 

-handleReflenaratefybflfmsgrReganereteMag) r 
•quBueTTmeoutHa ndlarftlm8dO U »n fl t an ce a . U6t? 



1m* 



<<Int9rfacQ>> 




Regenerator, init 

// Initialize my config variables 
conf ig . get (regenHost f REGENERATE HOST 1 
config.g e t (rB genPort, HHciSStS) 



CacheManagar 



— -^J -componentForwardPath : String 
+cm J -modules ; Map 



^'» a ^-«..tQ^ i ^tH^ OT(TOeneTiteouraanalerJ . ■ 

. STaS^^^— -II 
regenerate (iMd) ' 

return false • er ' 
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if tpltMa.regeneratePolicy Always 
return true • 

Date now a new Date 

Date lastAccess , Filelo.lastAccess (path) 

retoom (Wgrtltae - laatAccess .getTfcne) / MILLISEC PER.SEC <= 
tpltMd.getBegenerateLastAcceasedlnterval 

regenerated twa, iMd) 
* 

regenerate (iMd : InstanceMetadata} 

. tMd = maCon.findTeinplate-dMd.getTeaplateKey) 

regenerate (tMd, iMd) Tr*'- , 



regenerate (tMd : TemplateMetadata, iMd , InstanceMetadata) ' 

iMd.expxrationTlmesfeamp + • REGENERATE_jaAX_TIME ■ TO REQUEST 



RegenerateMsg ' 

tMd : TeinplateMetadata 
IMd ; InstanceMetadata 



The following roerhod handles the RegenerateMsg: 

^RegenerateMsg (regenMsg s RegenerateMsg) ' 
tMd = regenMsg.tMd 
IMd = regenMsg.iMd 

//. Let modules add their parameters to the request 
request ° new VgnHttpServletRequest . re ^st 

f0r ^? 0 1 ° 0dUle ^ U f 9 in iMd - e ™»erateModuleValues, 
return 

" re?ura le ' fillRegenReqUeSt{reqUeSt ' «o*dWalue») 

// Sec hoar & port 

if regenHost. length, u Q, 
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else 91 " h ° 8i & P ° rt fron ^.getRegeuHost 
• host b regenHoat 
Port = regenPort 
request . setServerNane (host) 
rstjueat . setServerPort (port) 

else ; °«eaaerc x -vsn-site-ad', • tMd.getSiteld) 

request . setDRI (tMd. getPathAtPg) 
if iMd.isComponent, 

// £ 3=sm25^ : js «- 

// compaaeat output to natch S^SS*^ 80 the ?G can traascoS tS 
// conflponent. natch the encoding oj the template' that called the 

l( Add. is-regen header T»m« v ., : • 

/' gift-*- ^ ttESSt ST 1 " "'«* 

// xt and knows to jiot servp fi, fl «*wsraed in the request to the pes pp cL-„ - 
. ^est-addHeader,.^^ • PG - Seea 

//• Add identifiers of diff«r«,l 

// Add param-hash header so fiH.. ^' 
send request 
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/ / differs from normal vlar*™^* ' ^ 

// between different t^es oT2p ir S * race ^ions 

// races conditions only occur^J the t^f " ™ ***** instaace - " The 
// regeneration. For example, if a ZtLt^ Z** * '^^rate-policy causes 
• // already being regenerated due to a S!?" T** f0B 811 insfcaBce 

// not pick up the new clear-Sche id « Ss nf*'*' ?! regen ^ or "BT 
// regen placement's clear-cached is Jl pla ^ ut hand ^r sees that a 
// whereas an old normal Placement is iSt'^ f* 3 the generation, ' 

placementMsg^etRegenlhstanceSSd^etK^ dlSCarded - 

// Timeout or L"Sr! W^r^° ldTena?late ' 
• iMd.setState (Invalid) ""^^to ^ instance. 

// Handles RegenerateMsgs that time «..<- • 

gueueT^outtandlerftimldOutSstSer' S.Jf* W". .; • 

iMfi.eetState(Invalid) 1 UsC) for eac * ^ in timedOutmstanc.es' • 

iMd. update • 

o*.i Regenerate request 
.^™^™.**. h os t .^ 

^^S^S?4S^^ W«h 0 SEratn e rttoHn P. so 

instance metada* ft. bI« hSSteS ™ SE Ue «* ,Son i! <° » «» 

which all regenerate requests ^ ° Vemde ^ behav * r b Y Reifying a hostrport to 
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0 . r. 

9 Configuration Space 

91 Variables at the CM/cache level 
5.1.1 Public variables 
9.1.1.1 HOST 

The name of the machine CM executes on. 
9JLL2 PORT " 

Type -integer " 

The port CM listens to for HTTP messages from AM and PG. ' 
Min-I 

Max-65535 • •'"*•. * * 

. 9.1.i:3 . .CACHEJROOT ' 
•Type -Siring- . . . ' l 

The fully qualified path' of the cache's directory root 

1 • • . * * » « .* " * ' * 

9.1.1.4 / (XC?DU^E_toEN5rON_S$I ' 

^wSSSS* " wffl bs taved v** — tom 

% SSKS? " ' 8 ^ MW ^ G"— -* <° W» - Mowtog procedure when 

1. Configure your web sfcrverto parse the new extension 

2. Cta^CACBtfll^^ 

..3. Execute VgnCDS.cIearCache(new StringO {"/*"}, null) 
4. Configure your web server to not parse the old. extension' 
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■ ? . < 

9.1.1S CACHEjflLEJEXTENSION_^0 _SSI 

Default -ygn. • 

Dynamic change - start using the new value. 

9.1.1.6 C0MP0NENT_SSI_PREFIX 

^^Svarv^lgoawayfcroneofthefoflowingieasons;. 

For IIS web servers, the default .is "exec CGI", 

For other web servers, the default is "include virtual": ' " ■ . ' 

9.U.7 P60L JIZE 
Type -integer 

Min-l " • 

Max-256 , 
Default - JO ' 

9.1.2 Hidden variables 
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not documented. But if created, codeTfl* L m^^T* created, and they are 

necesaaxy.butKd^^^^ 

9.1.2.1 CL£ANUPJ»EHIOD 

Type - integer . . 

The instance deleter wakes up every CLEANUP PPPTon 
( m . 0 • ^ANUPJ^ERIOD seconds to delete invalid instances. 

Max -2592000 (30 days). 
..Default -600 • 

91X2 . CLEANtJP JDELAY 

Type - integer • r 

S^^^^^S^^i^^ tod «*• file) whose 
metadata has time to propagate to all cli^W^fJ dda y^ sure the instance 

should occur within a secon^^ 6 . 

Mm-fl • OTaveJy ^ aco ^ Ieo fordersofmagnitudelarger. - . 



Max -2592000(30' days) 
Default -300 

9.2 



Variables* the S tage:CDS or stage:CDS: S ite level 
9X1 REGENERATE_HOST 

The name of the machine CMsmrfc r„r™„ ' . - 
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9.2.2 REGENERATEJPORT ' 
Type -integer 

^eportCMsendsregeneraterequeststo. Used when REGENERATE JHOST is not blank 

• Min-l 

Max -65535 

9.23 REGENERATE.CONCURRENCY.LIMIT 
Type -integer 

Min-l ' . 

• Max.- 1024 . 

• r 

DefauIt-2. . ' 

- 9.2.4 REGENERATEJ^X_TIME_TQ_REQUEST ' " 
' Type-integer ' " '' • ' " .'. . , . "' 

Min-l .'•'*' 

Max -3600 . ' ' ' 

i 

Default- 300 ..." . " .» 

9.2.5 REGENERATE_TlMEbUT 
Type -integer 

The maximum number of seconds CM waits for th* . • 

timeout occurs, the old instance is inva^d^ fftbe ■ 

Min-l . 

Max -3600 
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•Default -60- 



C. 
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10 Policy definition file 

• SSS SSw'f y W ^ ^°^y "9"*** ^ derations (rather than distinct 
booJeans hie ContentChange) to simplify selection of the useful business cases. • 

10.1 expire-policy 
One of the 4 following values: • 
I. CwtwlcaMi^lyMdiftB^ 

2 " 2SS^^ ifCOto,Ckg ^ fef ° riwlicyl TemplatemuW ' 

1 V^mT tf t (t£ f? Iate d06S " 0t U8e COntent API or uses content not managed by 

VCM) and (content changes on a regular basis). 

4 ' com^f n^V^ f ^ ^ Iate is U * dated or c]eared ™ «*« «*• API 

; VC^Q and (content changes under control of another template). The template that initiates the 
content change must use Clear cache API to expire tempi that use the new! S 

10.2 empire-period 
. stillheed.to think about: • -v , 

• ^^nibtoatfons^ \ 

• daylight savings time changes > 

«Pire-pcnod is ^ specified in the foUpwmg subHslements: niinute, hour, day^f-monm month-of-vear ' 
and day-of-week. Each sub-element is a string conforming to the follovdng syn^^ * ' 

* | elementlist J everyXUnats 
*=a]J legal* values 
elementlist = element, element, ... 
• element ^mraiber | indusiveRange 

inclusiveRange = number-number ' 
eveiyXUmts = /number 

Days may be specified 2 ways: day of the month and day of the week. If both are specified, they axe 
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additive. 

Hie tiine zone is the local time zone of tbe machine 
Examples: 



running cache Manager. 



minute 


how 


0 


8 


0,30 


* 


0 


0 


12 


* T 



day-of-month 



aonth-of- year I day-of-week 



10.2.1 minute (0-59) 

Minutes of the hour that the template expires on 
Legal .values are 0-59 
• Defaults 

. 10.2.2 hour (0-23>. • 
Hours of the day that the template- expires on 
Legal 'values are 0-23 
Default -0 

10,2,3 day-of-njonth (1-31) 

Days of the month that the template expires pn 

Legal values are 1-31 

Default - + 

10JZA month-of-year (1-12) 

Months of the year that the template expires on 

Le^al values are 1-12 

Default - * 



result 
every day at 8:00 am 



midnight every 1 st and 15 th and 
every Monday 

every 2 minutes' Monday-Friday 
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10.2.5 day-of-week (0-6" with 0=Sunday) 
Days of the week that the template expires on 
Legal values are 0-6 with 0=Sunday 
.Default-* 

10.3 regenerate-polky 

One of the 3 following values: 

1. Never 

2. KLastAccessed^^ 

3. Always 

10.4 regenerate-last-accessed-interval 

T 

Type -number of seconds * 
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11 Plug intb generic framework 

HtlpAglet and cX^^^^^i ^ COnfig ^ 8 V ^ ent ^ aglets: 

• /placement , com.vignette.cod.cm.PlacementMsgReceiYer 

• Zmeta.iata-com.vigqettexodcm.MetadataMsgReceiver , 

~A^^Sf* " ^ ** h ^ * - "own .etnod 



ii.2 Message receivers 



a • 
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12 Monitoring attributes 

■ ^ ueue instrument the current # items in the queue. 

• CM 

o # normal placements 

o fregen placements ' , 
o .# instance expirations caused by clear-cache 
o # instance expirations caused by template update 
o # instance expirations caused by record update ' * 
... o #instanceexpirationscausedbyperiodic expiration ' 

o # failed regenerations , ' 

o #regenerations that time out waiting to be sent to PG 
o # regenerations that time out waiting on PG response 
• p list of active metadata clients ' 

P list of disconnected metadata clients (perhaps)' 

• PG 

. o time to generate . 
o # cache hits 

• WSplugin 

•o # cache hits 
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13 Open issues 

i ' 5 ' jf WMlmm/200of PaS€ regenerad0n probably ™w worked correctly on 

GetHleMormabonByHandle. The former can return a stale value. What does java use? Tryto ' 
reproduce the problem with COD. r^uuwjdva.use/ xryto 

13.2 Vicket8233(CUST: CMD should delete directories when template is 
expired/deleted) r . 

men zr«^-msg.tempkte^ n try.template^veut is delete, make sure to delete the 
cacheRoot/pathAtPg subdirectory afterdeleting aU the instances.. 

13.3 Clearing WebseryerLookup table when WS path to PG patlvmapping changes 
■ Do we add a new API command? Probably not since.it should be extremely rare. : 

g» CM listen for an HTTP msg sent to placement port by some standalone utility? From email from 

♦ " ' * * * . 

• JX^"^" 0 m ° pti0n h ae metadaia too1 to cIear ft « or perhaps clear 

• • • SlnV^^ ''T" 8 - m ° « Uestion becomes - how *» this Set xSSr ' 

cached page. A subsequent placement will be for an existing cached page - but, the CM' should 
. ^ays try to update ^ 

2tS^ wi^iST ^ S " PG ? t maPPing * W£ jUit add 311 ^ ele * e "tin **CDF or PDF 
to specify ws-path-pattem. to pg-path-patteni mapping? 

13.4 Exploded deployment and CDF external to .war JUi deployment order problem, ' 

teltw°r makeS ^T^t toe CDF "* a tem P iate t0 parameters, and if the 

template changes are deployed before the CDF changes, there exists a potentially SSSZui 
wmdow where requests containing one of the new p £ameters will pcotoS dSSSSS new 

the new parameter will return results that are only produced when the new parameter is^Sent 

The solutionis to make sure the CDF changes are deployed before oryith the template changes. , 

™™!T™ k r? m ?uT ter^plateto remove sensitivity parameters andif the ' " 

CDFchanges are deployed before the template changes, there exists a potentially significanTtirning ' 

53 'Vignette Corporation Confidential 



f J 



■ request, without the obsolete paWt^^^l*' *^ S 9°*"°W 
parameter is present ™ retum re ™ B ta TO ""ty produced when the obsolete 

the 2 prev;<J bullets are ioor^Me ^ ' Mt ^ Saroe ^ «« for 

The solution for all 3 caaesis to mab fi^^nnT? ^ 

This najui^ van ^S^f^f "** 48 ten * late cI ^" " 

too a single deployment. Thii ftaffiS? £ 1° ^^0^ ^ ^ 8 dc P l0 ^ W 
ft^dworfclitePerf^ 

• 13.5 Too many cache files in a directory 

problem. . P * fin& Howe V er . w will continue testing tp see if it becomesa 

13.6 Configuration bootstrapping ... 

Weneea^goreoutthebaiicsceia.ost ^ ' 

13.7 Configuration teardown 

Wh-isthe^enoefor^g^^^^ ^^^^^ . 
13.8FaUover 

S^P^^ ■ 
andcacheoverfromtheprimary. • iftebac3cu P ^^dslcepycatthefl.take control of the repository 
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14 Closed Issues 

* • • * 

also called by a cached template, then bom ^tS^TL ♦ ! I com P rae "te is cached and is 

fa the cache-. If a broZ^y^S^S 30(1 0UtpUt be cachcd 

from the cache. 7 q ttB cora P onen t. our plugm should not serve the request ■ 



The solution is: 



thcofj-crvrfftout. . '^ CeSWUs ° mte ^^ 
■ M^mo^ffen Boston v HetHef ******** ^ , 

• else . . a ' .. ■ t t 

out.jprintint "another* ); . 

Do-the following things to males sure output is properly cached.* 

• Add an entry for each URI to the CDF. 

- U^coa-pooe,, upj hm,^ with HUpSccvWe,^,.^^^ ' \ 
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■ .? ■ . ( . 

This also allows the direct and component instances to have differ^ caching parameters. 

14.3 Vicket 57778(page regeneration doesn't do what U says on the tin..) 

Ss* £?S£I^^ I T ' m D3e ? My - they Sem fxom ™ AD*, last 

14.4 Record state guid 

accepted (events 8-9). . P ( 5) a P la£ement (went 7) that is later 



deployment #n, rcdi 



.J recox-eupdatp(2i, rcfll) 




. r. 







Inst ad 




cache' 


4 n 




* n valid 




8 a 






— 
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. 14.5 Clear cache ID 



rape condition 



.ad 




iart itid 




cache 






0 (*) Valid 




7 M 


1 1 


1 


9 (y) invalid 

.... ' | 






PG2 



rod 
10 y 



lust md 



cache 



.ndproeesaes thephee* £^^22^ M ~- ta *' , r dear0aChe - 

* 4 ■ 

14,6 Instance version 

which WS ox PG^Te^ a eaefe ffle ^LT™ a " 0Wei 10 *«e h, . wfadov, in 

for both versions (otherwise it wmlSfcTTSr »» the mstance raetAdata will be the same 

content irocWl^^^^ ' 
version, of conLst • be Afferent /or same maumce when generated with different 

*^*™^e^ 
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™^TS ?2$'J£ S "*? ? for " iMa,ce « multiple rows (probably 2 at 
14.6.1 Regen placement overwrites file 

The mismatch occurs jn a window that starts when the cache file fs written with data from content 



deployment $ n , rcdl 



AM 



1 rwcDr<Kjpdate{n, rcdl) 




n-1 
2 n 



n-1 
8 a* 



Valid 
Keg en 
Valid 



7 a 



4 regan(n) 



14.6.2 Normal placement overwrites fUe : 

i^LTc^ occurs/because WS andPG serve from the cache when an 

tn^stance state to Invalid Tnvahd would probably propagate to WS and PG well before the new 
version of the cache file written (event 17), so there should notbe a problem in theTo^ cal ' 
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deployment foi, rcdl 



fl. recorapjpdatalji, rcdl) 








■* last &d 




cache ] 


a-l 

9 k' - 

• 1 




7 a-1 Valid 
13. a- yalid - 




a? a' 




— T 11 







14.7 Metadata record locking . 

i^^^°to t ai8TOj S c^^n)ehaW(M -MeS c OBCDnency wifiiin CM requires in-memory DB record 
14.7.1 Record update after CM starts Writing cache file . • " ' 

iSd^^%^^ e f^2£f placement tries to upduLe the induce . 

/(event 7) de^ fcS^^S^? ^ f ** (eVent 6): ^ ^^tance metadata. update 
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CM 




deployment *n r red! 



4 recoxdt&>date(n, rcdl) 




3 placement (n-1) 
9 regan placement (n) 



PC 




ws 


« 2— _| 




ra 


1 1 


n-1 













, version 

n-1 Invalid 0 
7 n-1 Regen 1 
U n-1* Invalid 1 
11 n Valid 2 



5 

la n 



n-1 abcdLO.shtml 
n-1 abcdl: 



abcd£ 1 . ahtml 
ai»ccU2-ahtml 



8 regen (n} 



ra o Records t ate table 

to^ctioi^o; 81 ^^^^^^ transaction isolation levels TimesTen holdsall locks for the duration of the : " 

lection updating a row blocks outreaders and writes until ttetramaction commits' 
.-.Any tranauraonreading aj-ow blocks out writers until the transaction coSs 
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record update (event 6): 

ilHiSE^^ rcaupdt! ff etcdsld) 
mdCon. commit • 
write'instance metadata (event 7): 

if instance should be regenerated 

inetMetadata' state =S 
else. * 

ae ^? eta f ata,State " invalid- 
••• set other instance metadata fiei** 

inatMetadata.newJtecord SldS 'V r 

indCon. commit 

if instMetadata. state •== Regen 

eend regen to WS ' 
else if instMetadata. state « Invalid 

delete cache -.file ^nvaiia^ . ... 

Since the update occurred 1" the nitm* * ■ • ' * : 

written in the Valid state, then the subsequent SSdSJ^ ^ 1 • *» iDstan ^ would been . 
its regeneration. . ^^^t record update would have either invalidated it ot started 

14 .72 Concurrent placements \ 

same is 

foUowang scenario ehows-that a similar ^SSSS^S^^f^ c,earCache ^- The 

. . tnrng can happen wth the.veraon, but-the solution is different. 
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I. 





4 — _1 Placaaoat x 




.CM 


-£ ... 2 placement- y 




4 




PC 



inst aid 




KM state 


version 


3 x&y r 




5 Xfiy rAtf Valid 


0 



cache 
4 x&y abcO^O.Bhtanl 



(event 4). ' (evrat 35, 80 both deade to w° te version 0 of the cache file . 

14.7.2.1 Problem 1, concurrent open and write to the samefileby different threads In the reprocess 
Here are 2 basic options for solving the concurrent file access problem: ' . 

1. Use file locking to serialize file access 
. 2. Use some other mutual exclusto . '" 

control we nStoSLS of ■ 

. thatmaydoSck 1^ •• 

. 1. Jf the failure is "duplicate record," just ignore it. 
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14.7.2.3 Adding a new state 

with « to ^.3 *i P'^MsgHmdler write the instance metadata 



cause 
state" solution. 
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14.8 File locking 



14.8.1 Vtf ys COD file handling 
• • V6 : . . 

o Expire without regen: rename cmlhtml -> curfJrtml.bak 
■ NT - uses a quite complicated method MTmv: 

• tries MoveFiJeExW; returns if ok 

• open src file & get a file mapping; share delete, read, & write 
•• open dst file w/truncate, share delete; read, & write 

• if dst file not found, openw/create new, share delete, read & write 

• for each chunk of src, MapViewOfFile, WriteRle chunk to dst ' 4 

• close src file & mapping 

■ •• open existing src file, delete on close, share delete, read, & write 

• close src • 

» Other -VU_OS::rename. 
•o Expire with regen: rename temp.html' fOcurJJitml t . 

Calls callPageDaemon in cKents/cormon/generatePagelmpLcpp w/flae that WS 
writes cached pages 

• VU_OS_rename(tmpFile, curlKle, removeSrc=T) 

o. NT-CopyFileW(taprae 1 cur3FUe ) fdlIfExists=F): 

•wunlink(tmpFile) 
o Other - rename • 
• •- VU_OS^unlink(curl.html.bak). ■ . 
••• ■ ' . * o NT-._wunlink ' 

'* o Other- unlink ' 

* COI> .. ' ■ ' ■ ' 

, o . Normal placement: 

■ ^Wpfc threads can.concu^^^ 

process will not have the file open, because it's a new version 

ZZ^^f 1 ^^ WiU toUSfi java - ni0 fiJe Iockin S when ^ting 

V • t BL l a d threadlS Wocked mta ltaon b closed - 2 8d bread's open then 
succeeds & 2 create truncates the file & rewrites it 
o Regen placement: 

since there will only be J Tegeh placement- 
o Expire without regen: 

• mstonce deleter thread deletes file. jaya.io.Fi]e.ddcte fails jf dst is open by 
another j vm, but it should still do since deleter doesn't delete instances tin enough 
time has passed to be assured no one is still usbg the instance. ' • 
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m*2 Vicketi23*5<^fiie. to ^^ 



Original Message 

Prom: Rajkumar, Isaac 
Sent: Friday, January 18, 2002 3:12 PM 
To: Afshar, Jamshid; Caldwell, Davi'd 
• Subject: RE: AFM and COD 

I was just commenting on the reDroduHwt^, 

•COD (like David hah done) ^Ss 7 " We need * to kee P of this for 

-Original Message. > 

Prom: Rajkumar, Isaac 

Sent: Friday, January 18, 2002 3:09 m ■ ? 
To: Afshar, Jamshid; Caldwell, David 
Subject: RE: AFM and COD 

ocemraca .that i. «. r ^uS^le. ' " l0O,!3 " to chl « " » Pretty iar. 

. — : — Original Message 

Promt Afahar, Jamshid • 

'Sent: Friday January 18, 2002 3:04 PM 

To: Caldwell,- uavid; Rajkumar, Isaac 
Subject:- RE: AFM and COD 

saS^bleJ^e^^oSemrWMg LTE^^V??^ What a PP ears to be the 
MarJe that QA i« abls ^reproduce it JfifS C £ iCi ^ Als0 ' 1 jtist heard f *°* 

Instance version, instance stale, and the instance cleanup thread obviate this problem 
14.9 WebServerLookup CwicwTency 
14.9.1 Concurrent placement-msgs 
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14.92 Concurrent template delete metadata-msg and placement-msg 
1«3 ancurrent template add metadata*^ and placement-msg 

transaction that inserts the wm™!, *I existence of Hie template metadata in the same ■ - 
• that adds thetoplatSiT ^ ^ ^ ^ ?» 48 same faction 

14 JO Redundant metadata-msgs ' * '." 

1. Add objectld and modCount columns to template metadata " 
' 2> e^o^ed ^n^r d ^Ti! 15 ^ ]ate Update/delete metadata-msgs to CM (In ■" 

3. CM.MetadataMBgHandler.addTemplate: * 

a, if tMd exists, return • 

b. add the template rnd, etc 

4. CM.deJeteTemp]ate; . . 

a- if iMd does no(exisl, return 

b. iftMd.objectId!=^^ 

c. delete tfce'tempiate md, etc 
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5. CM.updateTemplate: 

a. if tMd' does not exist, return . 
' b.- if tMdobjectld 1= radMsg.objectld, return 
c. if tMdmodCount >= mdMsg.modCount, return . ' 

e. update template rnd, etc 

' 6 ' 

,' 7 * t C en?p?S^ , 

. before the deletion, w*en it comes bac* IwrnTSS^R ^ ^"^d. Tfmm goes-down 
objcctld, it appears to be a new ^essS E th SSf !? ? * Wgh m ° dCoUnt « * no 
the template or war was re-added! 0UDt SaVBd m fte mfetadata wen * back to 1 when 

. , 

- ; app.war created with objects, modCount=l. It contains: . ....... ' ' ' 

a-jsp with hash=a • ' 

b.jsp with hash=b 
This results in the following metadata: . 

ajsp with objectId=m, modCount^l, hash=a • 
. b.jsp with objectId=m, modCount=l, hash=b " ' 
app.war updated with objectId=to, modCounte2. It contains:* 
. stjsp with hash=c •- 
b.jsp with hash=b - 

•This results in the following metadata: • - 

a.jspwithobjectid=m )m odCount=2,hash=rc- V 
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b.jspwitbobjectId=m;modCount=l,hash=b ' • 
Now suppose anew AM is added 

app.war is created with Dbjecad=m; mo dGount=2. ■ It contains: 
ajsp with hasb=c 

b.jsp with hash=b * 

AM sends a metadataMsg containing: 

ajsp with objectM=m, modCount=2, hash=c 

b.jsp with objectldsm, modCount=2, hash=b 

• If CM just looked at objectld and modCount, it would unnecessarily derail' of b.jsp* i 



instances. 
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nacement protocol and placement message handler sections' 

are fairly solid They are reviewed and feedback is 
■ incorpora ted Other sections, beware. 
Added instance state diagram. 
Added note to regenerate:iast-accessed-mterval about 
. possible WS in memory caching problem. 
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ws-only. ' • 

Added concurrency architecture. 
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.to .war file deployment order problein. 
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6/12/02 . 
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. length headers in each narL 


0.10 
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. Corrected SSI offsetin scenario in PlacementMsgHandler ' 

section. Added corresponding note in -pseudo code. ■ 
• Made CacheManager diagrams only show fields Sc methods " ■ 

• . used by calling handler. ' 

Added CacheManager diagram to CM section -with all fields 
andmethods. 

Fixed inconsistencies in a invaHdateC^RegenEachlnstance' 

andpolicySaysRegenerate. ' 
Component forward path changes: used component 
. patbAtPg in SSi & regen request instead of fwd pain; , 

removed vgnComponent QS; added component regen ' 

headers. 

PlacementMsgHandler handles wsLookup even on duplicate 

placements; also sets inst parentUrlAtWs. 
Added dynamic-msg. 

Added closed issues - WebServerLookup Concurrency, and 

Redundant, metadata-msgs. 
..Corrected stage^cds-dg diagram and description in intro. ' 
Added conditional .transition to Regen state to Instance state 

diagram. 

Deleted minheap icon from concurrency architecture diag.' 
Changed pathAtWs to uriAtWs. 
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9/27/02 ' I David Caldwell 



'^S^^ 1 ^ Cache faJe when ««™t 
expiration occurs. ■ 

Renamed D1RJROOT to CACHE DIR. 
Noted that COMPONENT SSI PRFPTV ««n k 

CACHE_HLE^E3CTENSrON_SSI& 

CAaffi^ 3 XTENSION_NO_SSI - changed 
. dynamic changes; moved to cache-level 

^^^-^•^ed-interyai - reference WS plugin doc • 
-.i^fP^ofWS last access oddities. 
Added list of metadata .clients to monitoring section. 
Added open issues: "Too many cache files fa a directory " 

Changed d£ Id fa giteH 



nmctea veaion to match psf^ revisH)1L — 

Renamed dirRoot to cacheRoot 
Peleted dynamic-msg. . 

.^teSto a Ce ^ nt " rasg - tem P latB -P Ia =^ent to pached- 

M S™?1 w P laceffien ^g.dynamic -outer-template 

element before cached-template elemenL 
Renamed Instance parentUriAtWebserver to 
. outerUriAtWebserver 

^ S^ el ; SerVeriS request 
««S b0t ? 0U ^ tem P iates ^ components. Andfbr 
components, addx-vgn-stage-id,cd S .id, site-id, • 
c«nponent-path-at-pg;and response-content- 
type headers. 

P g for the regcn request uri. And for components, do not 
add stage, cds, site, and patb-at-pg headers 

thanJMS. AJso added Dald for Templates tate table 
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11/7/02 ■ 


David Caldwell 


Moved clearCacheOption argument to findTemplates call 1 

invahdateOrRegenlhstance: made inst undate a single tnmt 
Added CacheManager.getFulFflehame. ■ 

? earCacte M S gHandler .during implementation. . 
Updated RecordUpdateMsgmndler during implementation. 
tt j LU reau J matcn perforce version . 
Updated MetadataMsgHandler during delete & update 
_. ' implementation. _. • ! 


16 • 


11/11/02 


David Caldwell 


" \ 


11/20/02 


David Caldwell . • 


Changed PlaceroenlMsgHandler to make SSI & in MD 

component offsets be the offset in the original output 
rather than the offset in the cache file j 



8 • 



Vignette Corporation Confidential 



■z? 



1 Introduction 

stage i_ 6pntent de}h/ery ^ ^ 

1 " . ' 



cache manager 



web application. 

A^iaasetofidated^^^..... " ... * A CDS 18 ^1 stage. 











employment group 




+ 




+ 


•, s/te' — _ 




• • Aw ^ ap P^ on S^ratea a single site- 

A template is designated as cacheabfe J™ Ar becac ^'" 1 or m0 re caches. .• 

deployed to the Application Manager -5£tK JffTSt? 6 Cache ^nition Rie (GDP) ^ is " 
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Regeneration is th,e automatic re-executi™ nfawi.* * j 

Regeneration is triggered by eJnfaS? f t0pr ° dUCe a new version of a ^. 

request I-rametai^ wflJ - «* cache-sensitive" " 

/J Instance state diagram 

CMcKentswmonlyservefrom.thecachewhenthestateisV^ . 




«» 'placer**: [concurren; expire] ™, w 
proc,M " ■ fca»B»l»tft or rteartd^plqynBat or 

2a thiji mm, Lbo dcv inse-nc* U cr*ac«d 



1 t: 



10 
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2 Concurrency architecture 

wI° U T w g * a ? am * 0WI * XBad> 831(1 Pools that process the basic CM taction " 



HTTP messages 



o 



Instance 
Deleter 




Message handlers that execute 
on HttpAcflon framework threads: 

metadaiamsg • 
. record update meg 
placement meg 
clearcache meg* 




ii 
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3 Protocols 

3.1 AM- CM m etadata-msg 

The XML schema of the body of the HTTP metadata message is in 

■.V^om^^etteNcoaAc^toadataM^ xrH, The XMLdocument describes the addition of templates, 
the deletion of templates, metadata changes, template changes, and simultaneons changes to a template 
and its metadata. MetadataMsgHandler on page 13 shows how the messages are handled. 

3.2 AM-CM record-update-msg 

The XML schema of the body of the HTTP record update message is in- 

AAco^\vi CT ette>d\c^ordUpd a teM Sp ™rt. RecordUpdateMsgHandler on page 19 shows how the 
•messages are handled.- • '• re • . 

33 PG-CMpIacement-msg. 

r * 
The XML schema of the XML portion of the placement-msg is in 

A.Wvi^ette\cod\cm\plaoementMs ^sd. The XML document described by the schema is 
embedded m the 1 part of an HTTP 1.1 multipart/mixed message whose subsequent parts contain the 
template outputs to be cached. Eachparthas a :Content4engfe header that specifies the # of bytes in the 
part. The XML document in the Impart describesfte template .outpjtfs in the other parts of the HTTP 
. message-and tbeir.wjeirelationsbjps. PlacementMsgHandler.pn page 2J shows how the messages, are/ •, 

•314 PG-CM clear-cache->msg " • '' ••'»• • 

.The XML schema of the body of the clear cache message is in : 
vl^omVipietteVQdNcm^learCachfMsp y sd. -The XML document described by the schema is 
embedded in either the body of an HTTP message! ClearCacheMsgHandler on page 32-shows how the 
messages are handled. * 6 ■ . 
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4 Message handlers 

4.1 MetadataMsgHandler 

Handles the AM-CM metadata-msg described on page 12. 

The flowing diagram shows the classes that process the metadata-msg: 

~>j 



MstadaiaMagHantflar 



^«a fi aM.ctat B M, 0 ( h «pB (ri y : i r p Ut 31m a m, conterrtUnfltMrffl ' 

-*nvaMa1eSBiWn8tano»()mtenaes3.1sq 



PBrfodteExpir^iten 



■$t8mplatdMeladat8Updal8(IMd:TsmptatBMBmdatal 
•?tsmplatBUp<jBte(1Mii:TamptateMfliad«a) 



•♦cm 



Cflchi 




-InvandateOfflspsnEacWnstanwOM 


CTejjiplaiaMetadata, Instancaalla) 



-polley8a)OT. 8 enoi B te(lpUMtf : T em p| alBM9WaK pi^tf™,, . b(jotean 
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. 3 ' . u 

The foDowmgpBeudocode shows how the metadata-insg j s handled: ' 

jyten buffer f'new *S3£t££? 6 ^ ° bj6C * ■ 

/ int by tesRead «' httpBody. read (buffer) 

String string = new string (buffer, "UTP-8-) 
StrangReader stringReader = new BtrinatL^sLi^ * 
. -tadataMsg = -tadata^g.un^^ 

String stageld = metadataMsg. getstageld 
String cdsld = metadataMsg. getCdsXd 
String eiteld = metadataMsg. getsiteld 

" tawliteEntry.ietTmplateBvent «= uDdat.-hlrt 

t 

addTeniplate (stageld : String, cdsld • • 

• tsntry : Te^SeEntry? *. - lteId ? Strins ' = 'String,- 

/ / Begin transaction 
mdCon. setAntocommit (false) 

<eZ«e .e^xxootap fapl.t... Btagela , cds;d , BiteJd ; ^ 

// End transaction 
-zn<2Con. commit 

• JDdCon. setAutocommit ( true) 

PeriodicExpiration.teiqp2 a teAdd(tMd).' . 

deieterenrplate (stageld : String, cdsld , string, siteld . strina 

tEntxy : TenplateEatry) ' • M " Strin ?< . . 
// Begin transaction 
mdCon > setAutocomniit i false) 
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f™ 1 " 3 5? tuu:p » - ta>tiy:sets a u>stp a 

if tamM ^.^.^^ retum .- 

•iavalidateAlllx&tances (tMd) 
mdCon . deletaTemplate ( tMd) 

«. ■ • * , 

delete WebServerLookup rows with t^ni 1-=, „ ... 

th ten© lat e» fl stageld, cdsld, siteld, and pathAtPg 

// End transaction 
wdCon.cojmnit 

mdCon.setAutocorranit(trus) " 

PeriodicB5piration.teinplateDelete{tMd) " 

^Sc^^^iV : ^ateMetadata) . ' 

instances . indcon.findlnstancesrtlld) 

for each iMd in instances, r - • . . 

iMd.setlnstanceState (invalid) * ' ' 

iMd.eetSsqpirationTijnestajtrpCnew Date) 

^ iavalidateBachlnstance (instances : tigt) 
for each iMd in instances, ' 

wJCon.setTe^latelnstaaceStatefiMd, Invalid)-' ' 
^pdatereinplateMetadatatstageld • str-<r,~ ' * M 

•• • . l «2S J strin£J < slteld : str ^ ; ' - • 

tMd » ^Con.get^latelletadatalstaoS rf, T< * 

°«a<seageld, cdsld, siteld, tEntry .getPathAtPg) 

2 jk ^ e i^:£s- meg if ■ ****** ' 

if tMd.getMetadataHash ~^SS^Sj^i^?; ffe ^* tok ' retU3m ' 

• Matty - g8tTea » lateIfe ta<3ata.getMetadataHash < return 
doMetadataQhangeftEritry, tMdj/ 

// if any parameters were added, 

ff ■ -> invalidate (no .regen) all instances 

// « -V no-c^rj"™'^^?^ «- palters 
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// -> just update metadata 

// 

II Note: here's an optimization that could be made to the case where sensitive or 
II ^ndent parameters are removed. Instead of deleting an instance X that uses a 
// removed parameter P, • Y.parainHasb = Z_paramHash - P. If Y_paramHa S h is not in 

// to^SX^' 8 ^ Lt *** renaae *'* CaChS fiie fr ° m ^-^sh_v.ext 
doMetadataChange(tBntry : TemplateEatry, tMd' : TemplateMetadata) 

// Begin transaction 
' mdCon.setAutocommit (false) 

// Invalidate all instances if any parameters were added 
paramAdded » fal9e 

for each nParam in tEntry.getTeztiplatelJfitadata.getModuleParams, 
fovmd = false 

for each oFaram in tMd. getModule Parameters, 

- if nParanugetName s== oParam.getName ' * 

' . && nParam.getOptions oParam .'ge top t ions, 
found * true - ' ■ 
break 

if . J found, * ' ? . " * . 

paramAdded = true 

break . 
if I, paramAdded, - . 

// 2rf^ef StanCeB tJiat U8c " aay flensitive or dependent parameters that 

for each oParam in- tM.getMdduleParameters, ' 
found e false 

for each nPa*am in tEnfcry.getTe^plateMetadata.getModulePirains, 
. if nParam.getName oParam .'-getName 

&& nParam.getOptions a S oParam. getOptions-, 
found = true 
break • 

• . if 1 found && oParam. getType is no-cache, • 

removedParamList. add {oParam) ■ 
if J removedParamList.isEnipty, 

instances » nidCon.getrnstancesUsikgParains {tMd/- removedPar^mList) 

" ^ t8 ^JSlS? T ^ in associated tah ^ like ModuleParameter 

and TemplateStateJ given 1 the values in 'tEntry ' 

// End transaction * , • 

zndCon . commit 

mdCon.setAutocommit (true) • 

if invalidateAll, 

invalidateAllInstances(tMd) • 
else. If J instances'. i9Empty # 

inyalidateEachlnstance { ins tances ) 
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dpTettplateChajigeftEntiy, tlffd) ' 

// Begin transaction. 
mdCon. setAutocommit (f alsg) 

•• tHaah = ^try.getT^lateMetadata.gBtTe^late^ . \'. 
tWd- eetTenplateHaah (tHash) 

. tstate - nev TfenplateSfcate .".*' 

tMd.updata ^-^"^^Metadata.ffatModCount) • 

// Get this' templates valid instant 

" f™*** been' been cleared £ J"e inst ^cea, since they've 

// End transaction' 
mdCon. commit 

mdCon. setAutoconnni t'< true). 

^^^ ^L^^ t instances^ 
updateBoth(sta ff eId : String, edsld . c < 

tMd = mdCon.getTefflplateMetadatafetaae'Td «, M , 

afetageld, cdsld, siteld, tEntry.getPathAtPg, 
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a* 25°" th t m ? tadata -»s9 if it's redundant 
if tMd == null, return 



if S2S332S i: KS£^-^ H : — ■ 
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4.2 RecordVpdateMsgBandLer 

Haacfles tbe AM-CMrecord-update-ms f described on page 12. 

The following diagram shows the classes that process the record-update-insff 



r 



RecordUpdateMsgHantfer 



•updateFtec W dStat a ( a t agBl d:Strinp. ods ldiStrtnq. g utd^g j 



-cm 



CacheA 


Manager 




-IrrvalfdateOrRegenEacWr 


istancB(Instancea:Lret) 



Regenerator 



-mgenerateftMd^plateMetadate, /egen Instance^ 
— qnerateftMdTemolateMatnrinfc, IMd:lnstanceMatad fl ^ 
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_ teMSg -V nmara ^lRecordat> <3a teM^(stri 0 gJi ea d e r) • • 

fS^' S * ag ! Id B ^g.getstageld 
• String cdsld * ruMsg.getCdsId 

// Begin transaction • 
. adCdn.'eetAutocbnaiit {false).' ' ' •' • 

. .instances = ^w^^S^S^itt'' S*' 8 ? ° r re£ren ' d 

. ; update^cordState-fstageld; cdsid; rutog. g,^ '. \ - ' / ' 

• ', // "and transaction . .' 

ndGoh. commit '•*'■'".. ' 

mdCon . setAutocommit ( true ) 

• rcdStatt.setGuid^Jdj ^^^^^^^ • 
naCon.upddte(rddStateJ ' " . 

rcdState « new RecordState ' 
rcdState.setstageld(stageld) 

rcdState. eetCdsId (cdsld) . ■ 

■ "Estate. setGuid(guid) ' 
mdCon. insert (rcdState) 
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The foDowing diagram shows the classes that process the pla 



PlecementMsgHandler 



+pfocessPlacement(httpaody:lnpuStreani 1 bounda^Strfno) " ~"~ ~ 

H J ? J flSR '' IBC 9mentM3g, cacnedTpfcCachedTernplate) 



-cm 



-cacheRoot: String" 
-aJcheFfleExtenstonSsi : String 
-cachgBBBrfBnsionNoSsl : String 



CacfteManagar 



■wffBSsifstreamrCutputStrBarn, outputOffaetlm) 

U l a f m0ut P ulS,raam . outpiitqteetlnt, componertPathAfPorSfrlno) 
^o«60upjistream(stream:0utputstrdm) ' . • 

-«telateFDe(path:Strtns) .- • 

D5lfcySfly 8 He B enemta(tMd:T a mpfateM6tadat a , pathrStrino) --booJaan 
^erateflMATempJateMBtadata.lnstancewi , ' 
-/egBnaratBftMdrTefriplaleMBladafa. i MdilnstancaMeterf^^ 



Regenerator 



-pollcySaysHegan^fltedpltMdrTemplateMeladata, path;String) : boolean 
^e fl en e rate(tM*TartptetaMatadata 1 ragenlnstencald:lnfl ' 
_ , . I -fBoa"emte(tMd:TemolataMfltedAa MdjInsareaMetedafa} 

The Mowing pseudo-code shows how the pkcement-msg is handled: 

/> bttpBody points at beginning, of body ' 

processPlacetfent (bttpBody : Inputs t^eam, boundary • strina) 
strangReader = new StringReader (buffer, "OTP-8-1 ' 

■ - if msg.hasDynamicOuterTemplate, 

outer = msg.getDynamicOuterTeinplate 
outerUrlAtWs = outer . getUrlAtWs 

/ / Begin transaction 



// Begin transaction 
ffidCon. setAutocormnit f false) 

insert » true 



if insert, 



^ookup.setStageldCnS.gSsiageldl • 
waLookup . setCdsId (asg . getCdsId) 

wsLoo^.setSiteld^g^etSaSid) 



// End transaction 
mdCon. commit 

mdCon. settutoccamnit { true) 



■ • •••• :•• •' 

n^ata, httpflody,. boundary, cuterUrlAtWs) 
read OOF. • ,ne * Bo * : -^PUtStream, boundary) , Lit 



read — boundary CKLP 

^read -Content-lengtb: » 
read length CSLP 
•'Parse tbe length 



eat. the CRI,f that Dreceed* kv^ v * 
. return the length ' tte bo< ^-Part 



// -httpBody points at or * • 

Place (plaeementlteg s PlacSentv^^ ^"f* bo ^dary • 

*«pBody : X^ti^ t 2£ d ^r*gt , CachedO^plate, 
. aiy • string, outPHTriiK^ . 



t i-mjfj iRl.C f 

• i9t to » et ^e urlSWs even if the 
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. • 1 ' ^eger.Mft^vSLUE, cachedTplt.getOrlAtWs) 
// aSSt^tS'p.T' ' " C " " «~ or ^ PC 

return 



//. 3tt£?J£~?££!r after PG ^ 

//.metadata is being updated generates a placement while the template 

" r t e^' metadataHa3h " °*Wt\metadataHa 8 h, 
// Ignore the placement: pm 

// .the page. ^^^JS^SSTS^, 'ZS^ d f ^ *» generated 

// is being updated. ■ a W * placement while the template 

// ^STSuV^ rST-! K derate, the ■ 

// » placerant for an Im^T. vCT.r!, oe ! ™! ciear «*• another generates 

• " nnr^r^^^^^. .. .. . - 

// SiS'S'SSS.KP"- °~"" ea • r W n.r.n., ' 

catch OldPlacementExeeptibn 

If cachedlplt.hasRegenlnstanceld •* 
'' retuxn re9eaerate{UId ' cac ^".getRegenlnetanceId, 

// Ignore the placement if it's a du n 

// concurrent requests for the same Stance ^ 5 ccurs wh * n ' 2 «5a process 
// caches when a new WS or tQ cache is ° CCUrS at P re «cisting 

instance maxlnst « mdCon.get^vSionOf?™? ?f° W SyStem " c «P«*>ilities . 

• cachedlplt.getParaintelh) ersion 0^nstance(tpltMd.getKey ( 
if ma^lnst j« null && maxlnst.state « valid, • 

■ /'/' 33^±rS5c^ if. i„ ia ' ' 

ere can be multiple urlSws per template, and since 
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rv 

t 



// WS can be added after " 

// End transaction 
mdCon . copmit 

mdCon . setAutoconmit ( true ) 
. else // co^onent or dire it P S re ^ eat ' 
1 if outertirlAtWs l»nuU' ' ' 

return r * 

4 / rii f o^ h ro 9 : s / lMttoc ^^ ta&ta / ■ i™-* ^ ™ lon 

±f Jnaxlnat .1= null, 

version- -aaxInst.getVersion + i ' .V,\ 

. else^ " Ca,AePi ^tensioiiSsi ' 

«t = c m .cac* a piieE S ft e n slonNo6sl f ■ ' 

. return ' 
inputldx = o • 
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InstancaMetadata iMd = new InstanceMetadata ' ' 
List IncludeMdList = iMd.oetlncludes 
fot each include in cachedlplt, 

// Write the template output preceeding the include 1 • ■ 

. while (inputldx < include. off set) delude 

^ ffS f = DeW b y' te Iin^e.offset - inputldx] 

inputldx +c bytesRead 
stream.write (buffer, 0, bytesRead) 

isComponentlnclude = false X " • - . 

componentPath = null . . 
Include includeMd = new Include ■ r 

^i 1U ^1S :SetHaildler(include ^ etII ^«31er) 
includeMd . setoff set (include . offset ) 
Last ipList includeMd. getlncludeParameters 
£ or each param in .include, • 

S^SS^T*** iP B h9W ^^Parameter ' ' 

lp.setValue{par^m.getVa2ue) * 
' ipLiat.add(ip) ., f - ' ' ' \ ' .; • 

// <include handler="cod" offset=»l'00 u > ' 
• // . <param name=»type» value=» component '/> 

// </JS ,M, " ,p,Hwt r value^hosttbc/x.isp-^ 
if Include. getHandler «= *cod» 

else if param.getName == *path-at-pg*, 
conjponentPatlx = paraaugetValue 

components o isCompcaentlnclude & & componentPath i. null ■ ' ■ ■ 

// Write the SSI . 

cm. wrateSax (stream, include. of f set) 



includeMd. setLength (ssitength) 
includeMdla s t . add { includeMd ) 
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^l%^TT 2 eT tVXlt that fonows the last component' 

if bytesRead " ? :r fSr ' °' bu «". length) 

close stream; delete cache file- w «L 
.throw PrenatureEOF acne . tlie ' log error 
inputldx += bytesRead ' 

stream.write (buffer, 0, bytesRead) ' , 

cm . closeOutputStream ( s tream) 

U Start a metadata db transaction 
mdCon.se tAntoCommit (false) . 

iMd . setSta t e ( Ins tanc estate . VALID ) 

^ ™ * —data deployment whii e this thread ^ ^ 

rcdState * ^.£KS^££E^^»*I. ^ched^lt.getPathAtPg-, 
if. ^ - null II W^STRSS^^ 

H 2^^^.*^ .g-jpt ** metadata change).. 
. -^ete Pi le { pl a c ea entMsg r ™ 
return I» lac ^tMsg.getSiteld, path) " ° ls9 - cdsld ' 

tf If another thread' processed any of the f mi - 

// wrote the cache file: " Y "** £ol }™**8 events while this, thread' 

// *• template deployment 

// • * clear cache 

// • * record" deployment' 

else if tpltMd.templateHash 3= cached^! - , 

tpltMd. clearest™ . c » - OTp lt: . templateHash II 

//.^en the cached file already. expired,.' ' .' 

// If the instance can be regenerated 
if ca-policySaysRegenerateltlltMd pith), 

iL Set itS State t0 re 9en. 
elg ^- setState (InstanceState .REGEN) 
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o 

// Write last and ws lookup metadata 
iMd.setTeinplateKeyftpltMd.getKey} 

. mdModuleValues = iMd.getModuleValues 
List actionList '= iMd.geiActions'. r 

for each action In cachedTplt.getAction* ' * " 

Action actioi^ = new Action • 0 S ' 

actionMd aetHandlerfactioA.getHandler) 
List. actionMdParamList = acti^^ZZZL 

for .each parai, in act U^^SSSlT^^ ' " ' 

ActionParameter apMd = new jw^t* * * 
• ap^d. settle (paraTgetNaW) Action?ar ^er • ■ 
.apMd.setValuetparant.getValue) " 
actionMdParaiiList . add (apHd)* 

else // component or direct PG request 

iMd.setOuterOtlAtWs'CoutertJrlAtWs) 

mdCon.inaertlnstancefiMd; • 

// End transaction ' 
mdCon. commit 

ffldCon . BetAutocomrait ( true ) 
.// Send regenerate to WS 

Oft 
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cm- regenerate (tpltMd,- ijud) 



// Call with autocommit false ' 

WebServerLookup wsLookup = new Webspr™=.,-r „ ^ 
wsLookup.. setUrlAtWa (cacLd^T.getSSr 
^oo^p . setTe^lateKey (t^getS^ V 
'^!? P - Be r Sta9eId {PlacanaxtMag: getStageld) 
.weLookup.eetcdsidfplaceaeiitMsg.gefcSldr ■ 
^Lookup. setSlteXdfplaoeineatML getSiteid) 
ws^oJcup.eetPathAtPg^d.getPa^Pg)^ 
JndCon.insertWebServerLookupfwsLookup ■ - 
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4.4 ClearCacheMsgHandLer 

fiddles the PG-CMdear-cache-msg described on page ML ' 
neftDcmtag diagram shows the classes thatprocess the clear-cache-msg: 



CleafCacheMsgHandJer 



. CacheP 


Manager • • 




•InvalfdateOrRegenEachlnstancepMd; 


JampfateMetadata, FnatansesrUat) 



Regenerator 



-pottcySaysReflsne^nMd.TemplateMaladata, patteSttn} • boolean 
^generate^Tamp.ateMetadaJa, reflenlnstancaidrlnt). ' ' 

IMdrlnatanceMatedata) 



32 



Vignette Corporation Confidential 



Strong siteld • ccMsg.getSiteld 

cc Msg . g etci e arCacSSt x i, ,Sta0eld ' Cdsld ' 8 ^eldi, ccMsg.getPaths, 
for each tMd In templates, 

//Begin transaction " r 

mdCon.setAutocommit (false) * ' 

//Sert^^ . 

// in a S i. affle list you canVw- Bame typ<? « wa P them 

• 7/ fche S{U "e i»o<3ule name. fc have 2 °uter Map entries -.with 

// Get the valid instances tft k. , 

// they've already been beei cSje 6 ^- Ski * r ^en instances since • 

mdCon. commit • 

ffldCon. setAutocommit { true) 

-.invalidateO^enHachXnstancef^, instancesJ " 
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5 Cache Manager 

H* IM, dia^ shcwa to OcheMan^ c)ass dala mMnbers ^ " 



•host : String 
-port : Int 

-cacheRooi ; String 
•cacheR/eExtensfonSsI : String 
"CacheFlleExtenalonNoSfil : String 
^omponentSslPrefix : String 
-componentForwaitiPath : String ' 
-modules: Map 



— "*** ^ — - * 

^^Z^^^^^^^' W% Pa-Hashing, ve ra(o n:, nt , ^nsioncString) 
•wrtteSeJfstreamOutpuStmam.oulputOffeetim) r ' ■ 

-delat0Fne(path:StrJng) 1 - - 



Tht following pseudocode describes the CaoheManagermeAods: 

// Initialize my config variables " ' 

coafig.getfcacheRoot, CAOrsJob?) 

conf ig. ffet tcach^tens^osal, CCH^z^™^ SS1) ' . 
p.-uurn^.c.^id, cdsId , 6iteia , pathAcpg< versiQC( ^. 
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return cacheRoot + •/•Vatan.M x 

. ♦ > »«t» ♦ .,. ♦ f *«*•'•* , 

6X1 • - * SCrl W-»»l'aeof (version % 10) ♦ ... 

placement wortc ^ be aLStoSv^v™ ediat f ' , J 0r to » m .5. 
»«d to use JM (see eecta, I4 , 7A , j™ ™ Jf u»o can't dothat either, then we ^ 

threads irithe same pnx^s) 1 " Strait open and write to the same file by 

, ortiutstr™, of£set , lnt) ; te 

// Build the asi 

String ssi e «<j . 

# + componentSsiPrefix + . W . + , ?vmT . , _ 
/ / r 7vgnid=* + offset + n ff 

// Write the sal as bytes t-w ' * • 

stream. write (bytes) •' tt * n Teturn the #' bytes written 

return bytes-, length 

**teSsi (stream : .Output Stream. 0 f f aet 1 -* 

// Buhd the ssi ' ' ° mP ° GentPathAtPff - ' S ^ • -int 

String ssi ■ v <j --4 ' • 

" + •Pvgnld-! olfST^S?*' 4 *.* ^" * ^ponentPathatPg ', - 

iitlh^ ** t0 ly1:es f should- be ASCII w'.r 

Bytef] bytes- = asi.getBytes Cui^B-) Utf " 8 doesn ' fc h ^t) 

// Write the ssi as bytes k 

stream. write (hytesi retUr * ^ # ^tes written 

return bytes. length ... 

closeoutpumreamfstream :.OutputStream, ' 
deleteFile (path : string)" 

PoKcySaysRegeneritteCtMd • TemtVL^*, b , 

^^■^•^^•^^ iSSS' SJ = ^^etaderta, booleaji 
tMd.getPathAtPg, iM d g e tPar™» v ' , "^'WtCdald, tMd.getSiteld 
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• ^^^^^^^ ; 

regenerate {tMd : TamDlateM^* «. 

^ia.t« 3 «tt Bra tt, tMe , (tla _ m 

'/ Be»n transaction 
>»ffion. s , om toci» m it(fai ss , 

- // « '* : •' " cestata - RE,3,a ' - »«22it.. w,, ' 

iMd. update 

// End transaction ' 
mdCon. commit 

mdCon . s^tAiitocoOTiit ( true) 
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<> Instance deleter 

CLEANUPJSBLAy seconds. m Whost "?* ^ ^ U for more than 

Since' CLEANUP DET ay* * • 

rm not plannmg to j,, m ^ f * jT,^? ,? e ^.^oae someone still has it open. So 
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7 Periodic expiration 

passed to periodic expiration so it « SSffiSf , itS tempkte metadata is 

periodically, and if it's not already SSfd /S?, P - T®? 5 tem P Iate if it>s « * expire 
expiration for each periodically eStmrtate l**? * heap ° f * e nexr 
value is a liat of templates wboseTexTLS^ 7 ° ? e heap is *° next ^'^on time. The 

terminate expiration processing ftf the £3 f 2 ^ ? ^ 10 pes ^ odic 80 * c *> 

initialized After all aWSateS^ 

or another message is received an^Ww 1 ? ^ ^ ^ sIee * s ffl fee *** expire time 
followingproceSpg occurs: e *P*e *me occurs, an entry ia removed from the heap, and the 

tor ead> ttti In the entry, . 

S'SSt^T'""^ 3 ^^'^^^, ... 

addEnfcry(t, tMd) • 
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8 Regenerator 

toad poo.. Bach *mn*^£2gS2Z22£%' ? "T *»"■■«"« 

snessage and queue toeou, handlers J SSbrt m « 

Ihe following diagrani shows the classes that process regeneration: 



r 



Regenerator 



-regenHost : String 
-regenPort Mm 



CacheManager 



^ -componentForwardPath : 



■•poflqy8ay8ReoenBrat8(lpltMd:TemplateM8ta<teta naBi-si*™. . iTI — 
-moenerateflMAlnatancaMBtadata) ' P ' SM " 9) ' bo * an 

I -rafleneratepMATemplateMeladata, instancaldlnt) ' 

3»? te(Wd:TemP ^ M8kd ^^* ln ^atada1ai J 
-handJaReflenarateMsflfmag.-ResenarateMag) r 
I ^BuaTlmaoutHAndtorfUmBdOu.in^^.,., ^ 



•modules : Map 



String 




Regenerator . init 

iL??* 1 ^ 2 * confi 9 variables 
confiff.getfragenPort, RBOnrawaJ^S 



Taster. B^QueueTi^outHandler (^ueueT^outHandler) ■ ' 
regenerate (iMd J ' 

return false "ever. 



39 



Vignette Corporation Confidential 



if tpltM<3.regeneratePolicy == Always 
return true ' 

Date now • new Date 

Date lastAccess , FilelcJastAccess (path) 

return (nov^getTiaa - laetAccess.getTi^) / MILLISSC.PER SEC <= ' 
tpltMd.getBegenerateLastAccessedlntexval < 

regenerate^ tMd, iMd) 

regenerate (iMd : InstanceMetadata) 

. tMd = oaCoa.findTemplate-dMd.getTeatplateKey} 

regenerate (tMd, iMd) • . ^ 



™^f= e p md : Te ^ lat ^etadata, iMd : XnstanceMetadata) " 

'iff r^" at !f g J* tbe generator = 
iMd. expirationTiaaBta^ + . namMnSjSaSimjSS&igt 



RegenerateMsg * 

tMd : 'TeiqplateMetadata 
IMd ; Instancetfetadata 



The following method handles the RegenerateMsg: 

HandleRegenerateM 9 g(regenMBg : RegenerateMsg) ' 
tMd = regsnMsg.tMd ' 
iMd = regenMsg.iMd 

rL 1 i!^ ni0dUle8 „ add their P ar ^ters to the request 
request = new VgnHttpServletRequest * 

^J^ e7a i V f 9 in iMd -^erateModuleValues f 
module « cm.modules . get (aoduleValuas . getModu^eJanie) 
■ if I module. isReyenSuppoxted, uwnmeHanie) 
return 

" reKS le ' finReffenRegUeat{reqUest ' devalues) 

// Sec hosz & port 

if regenHost. length, u 0, 



40 



Vignette Corporation Confidential 



D 



^^et host & - port from ^a.getRegenHost 



host ft regenHost 
Port c regenPort 



request . setServerName (host ) 
request . setServerPort (port) 

' / / Set URi to outertJrlAtw* ^ m,„ , 

if iMd.hasOuterupiAtWebserver S * WS tier ' °r PathAtPg 

otherwise 

request.addHeader / u v-xTrT« ~ . * 

request. addHeader(»x-imn ^» L- ' ^-BMStageld) ° . 

, ' request.addHeadeJ 
else 1 x vsn - slte -i<3','tlld.getSiteId} 

request . setDRl (tMd. getPathAtPg) 
if iMd.isComponent, 

request. addHeader<«*iv»n — 

>Y Add. is-regen header- mu.i« v 

" Sift- - "- * "^L "^Lt S" 3 "' 1 ' -~ "» 

// it and knows to .not serv^ ^ " in the request to the PG on3!! 

. ^est.add»eader(^^^^ ^ PG - Seea 

// Add identifiers nf ai** 

tMd.getPeriodic^i^M ^•" e3q>lration - time stamp-, 
rcdstate » *?^f^ plr ^ tionTi raestampj ^ ' 

// Add param-hash header so fn^ a 
send request 

..to...^,^,,^^ . .. . . • 
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// races conditions only occur^eTSfV^!? f 011 the same Stance. ' The 
' // r f Sen ! ra "°°- i*ST If a clLrScL re9e e erate -P°^cy causes 

• // already being regenerated due to a f ^ f ° r 611 Stance that': 

// not pick up the new clear-cache -iS TSf S^' ?? regen or -V 
// regen placement's olear-cache-id is o5 ^ J*?""* han «31er eeee that a 

•.■•SSSS2£ESS pB -'»tai 

. // will succeed. So worst case the SL?' T Which "Wt. to that JG ' 
// — e is guaranteed that ^ - «—^ 0 ^'SijH^ 

iMd.BetState(lnvalid) 63 ' Liat) for ea =* ^ in tiaedOuttoatanc.es" 

iMd. update • . ' . . ..' .' 

8.1 Regenerate request 
' oecached*^^ request sent to'the hosfcpoit from the client request that caused the original instance to " 

instance metadata the Host header from 2 ™ f S? P SR The so,ution is t0 s *™ in the . 
which all regenerate njSSS behavipr b ? a hosfport to 
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9 Configuration Space 

9.1 Variables at the CM/cache level 
9.1.1 Public yariabjes 
9X1.1 HOST 

The name of the machine CM executes on. 
9JJ.2 PORT ' 

Type -integer * 

The port CM listens to for HTTP messages from AM and PG. 
Min-1 ' 

Max-65535 • 

i 

9.1.i:3. .CfCHOOOT , 
•Type -Siring . . . 

ThefnJIyquaUfiedpam'ofmecache'sdirectory^ro^^ ' / 

9.1.1.4 • C^CH]OTLE_^XTENSrON_SSI ' 

Z^^^™^™ V " U '- °«"«* » foBow ft. foliowfagprcc^or. w he„ 

t 

1. Configure your web sfcrvcr to parse the ne>y extension 

..3. Execute VgnCDS.cIearCache(new StringQ {"/*»}, nuU) 
4. Configure your web server to not parse the old extension 
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9.1.LS CACHE J<ile_EXTENSION_^0 JSSI 

Default -ygn. « ' * 

Dynamic change - start using the new value. 
9.1.1.6 COMPONENT^SSIJPRJEFIX 
" •. TOsconfigvar^lgoawayforoneofthefoIlowingreasons: . 

tor IIS web servers, the default. is "exec CGI", 

For other y^eb servers, the defeat is "include virtual " ' 

9.1.1.1 POOL_SI2E 
"Type - integer ' 

Min-l • 
Max -256 , 
Default- 10 



9.1.2 Hidden variables 

These variables control internal workings thai the 



customer probably doesn'tneed lowoay about, but 
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we don't want to hard-code. We would Hum h^n * 

that are added to template files fa SJ™ iSSr SOmet * ng Hke * e con ^ variables in V6 
not documented. But if created, ^^caDy created, and they are 

necessary, but hides unnecessary con^^^^ ^ OT VPS to alter the behavior if 

9.1.2.1 CLEANUP JPERIOD 
Type -integer 

The instance deleter wakes up every CLEANTIP PPPinn 
. mu . 0 • ^^^^-PERIOD secouds to delete invalid instances. 

Max -2592000 (30 days). . 
..Default -600 • 

9.1.2.2 . CXEANUPJDELAY- 

Type -integer • > 

A. . 

1 

When fee instance deleter wakes nri ft « i j i 

2S 1 ! n r be ^ .^^or more ^ ^^S^^^^ ■* Cache <&> ^°se 
metadata has time to propagate to all (fflS^S^S^: ^ ^ makes sure the instance 
should occur within a second, the default SSSSSS^^T' ** ,n W*» ' " 

Min . o ■ ™vejy set a couple of orders of magnitude larger. •• . 



Max -2592000(30' days) 
Default -300 



9.2 Variables at the stage.-CDS or S tage:CDS:sUe level 
9.2.1 REGENERATE_HOST 

The name of the machine CM ' - 
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^ .1. 

9.2.2 RJEGENERATE^PORT ' * 

Type -integer 

TheportCMsendsregeneraterequeststo. Used when REGENERATEJEiOST js not blank ' 

• Min-1 

Max- 65535 , 

92 3 REGENEJUTE CONCURRENCTJLIMIT 
Type -integer 

t 

Min - 1 

• Max.-J024 

r • 

DefauIt-2. , * ' 

'■*A4 ^GENERATE31AX_TBffi^Tp_REQUEST 1 •" 

• Type -integer " '*•'.' : " .' . ~ 

. ^S*^ a ^^^^'^^ in fa - cache wM c „ 

0 MJ„^ccis ta ^d al cd, Md «hc^S^SS^ Cmtte ^ Witototo ^ 
Mba-l ■ '' * ' 

Max -3600 

t 

Default-300 .. . ' . " .•> 

9.2.S REGENERATE_TIME'OUT 
Type - integer 

The maximum number of seconds CM waits fnr th* . • 

timeout occurs, the old instance* l^the^respo^toa^^ fftbe . 
Min-1 . 
Max - 36G0 
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•Default -60- 



C 
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10 PoKcy definition file 

■ tafeta^ re P™^ as enumerations (rather than distinct . 

oooleans hke ContentChange) to simplify selection of the useful business cases, • 

10.1 expire-poHcy 

One of the 4 following values: . 

I. ContentChange : yields freshest pages. Template must use content tracking AH. 

2 " ^SSS^'^^^^^ 7 ^ 1 ' Templatemustuse .' 

VCM) and (content changes on a regular basis). 
4 " ^ ce f When template is updated or cleared via clear cache API 

■ ^^Cwntentchangeaundercontrolofanothei-template). The template that im^ates the 
content change must use clear cache API to expire templates that use the new conte™ 

10.2 txpire-period 
. sfSH heed to think about: • .. -v , 

• ffl^alcorobmatibns^^ 

• daylight savings time changes < 

and day-of-week. Each sub-element is a string confqnning to the following syn^: * * 

* | elementlist | everyXUnits 
*= all legal' values 
elementList = element, element, ... 
• element ^miiqber | inclusiveRange 

inclusiveRajjge = number-number ' 
everyXUnits = /number 

Days may be specified 2 ways: day of the month and day of the week. If both are specified, they are 
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additive. 

Be time zone is the local time zone of the machine 
Examples: 
minute 



running cache Manager. 




10.2.1 minute (0-59) 

Minutes of the hour that the template expires on 
Legal .values are 0-59 
• Default. 0 

. 10.2.2 hoar (0-23)y 
Horns of the day that the template- expires on 
Legal values are 0-23 
Default -0 

10.2.3 dayrOf-njonth (1-31) 

Days of the month that the template expires pn 

•Legal values are 1-31 

Default-* 

10.2.4 month-of-year (1-12) 

Months of the year that the template expires on 
"Legal values are 1-12 
Default-* 



every 2 minute* Monday-Friday 
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10.2.5 day-of-week (0-5 with 0=Sunday) 
Days of the week that the template expires on 

Legal values are 0-6 with 0=Sraday 
.Default-* 

10.3 regenerate-policy 
One of the 3 following values: 

1. Never 

2. JO^ccessedWithinl^ 

3. Always 

lOJregeneTate-last-accessed-intervd 

Type -number of seconds * 



for 



5 0 Vignette Corporation Confidential 



1* 



11 Hug info generic framework 

HttpAglet and CmAgleY The o4nal of cSSSS COD config sets up a vgnagent with 2 aglets: 
.-*gmu«ad d ^ fi addition, 

• 'P^ment-.com.vignetteW^ 

• /metaoata- com.vigQettexodcmJrfetadataMsgReceiYer 

• / "^P<kte-com.tfgn^^ 

• /dearCache - coin.vignettexodcm.aearCacheMsgReceiver ' ■ 

rr ag s^^ - - ** * ^ ^ » ^ ae , hutdown method 

•CacheManagersingletoji. ^ m&a ^ ed ' thencan g et what they need from the " . 

11.1 Message receivers * 

. B. jmcq receiver invokes its corresponding message handler. ' ' .• : •;. . 
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12 Monitoring attributes 

collect by PO d^g iSS* «»» *-* " by CM could be. 

• CM 

o # normal placements 

o #regen placements ' , 

o .# instance expirations caused by clear-cache . • 

o # instance expirations caused by template update 

o # instance expirations caused by record update ' ' * 

... o #instanceexpirationscausedbyperiodicexpiration ' 
o # failed regenerations , 

o #regenerauonsmattmieoiitwaitmgtobesenttoPG 
o # regenerations that time out waiting on PG response 
■ p list of active metadata clients ' 
P list of disconnected metadata clients (perhaps) 1 

• JPG 

. o time to generate ... 
o # cache hits 

• WSplugin 

o # each? hits 
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13 Open issues 

1 ' J W^ommnOOO) regmeratU>n has P rMl y "»# worked correctly on 

' S^f^S^S file last access time via Win32KndFir 5 tHl e va ' 

%2£^^ Whatdoesjav,^ Tryto ' < 

' 13 J Vicket8233(CUST: CMD should delete directories when template is 
expired/deleted) . • 

cacbeRoot/pathAtPg subdirectory after .deleting all the instances.. 

13.3. Clearing WebseryerLodkup table when WS path to PG path-mapping changes 
• ; 9°weaddane W APIcon^ . 

%Z. °* liSt£n ^ mg ^ * Pl3cement P° rt b ? some st ^<™ utility? From email from 

• tSS^SSl * WC *1* B - ° pti0n b 0,6 metadata too1 10 clear *M* or perhaps clear 
. for specxfic web server path patterns. Then the question becomes - how does this SKSE'' 

cached page. A subsequent placement will be for an existing cached page- - but the SSESd 
. try to update me mapp^^ ' 

23.4 Exploded deployment and CDF external to .warjUe' deployment order problem, 
a a developer makes changes to the CDF and a template to add sensitivity parameters and if the 

The solution is to make sure the CDF changes are deployed before or^ith the template changes. 

^developer makes cbanges.to the CDF and,* template to remove sensitivity parameters and if the ' " 
CDFchanges are deployed before the template changes, there exists a S^SlJJJ ' 
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• requests without the obsolete parametovSl reto re 2* if S * subse <3 ueQt 
parameter is present I8ton r6SuJtS ftat « on] y produced when the obsolete 

the 2 previous bullets are LompaSfe . • * USe fte ^ ^ *« for 

jnto a single deployment This ftnS3tyhS use ^ST* ^ I** 8 d ^ 0 ^W 
It could work h*e Perforce* dting&stS^ * ' for ^oded deployment 

• 13.5 'Too many cache files in a directory 

problem. . ^ever, we will continue testing to see if it becomes a 

13.6 Configuration bootstrapping 

w.^^^^^^ ;;. • 

13.7 Configuration teardown 

What is the sequence for tearing down COB components? We need to figure out the basic scenarios. ■ 
13.8Failover 

and cache over from the primary. ■ ^ " d sIcepycat thol taie cw "rol of the repository 
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14 Closed Issues 

also called by a cached rm^S^S^^T^J 0 ^ ° f com P^ is cached and is' 
fa the cache: If a ta^^^^?^ 331(1 *» output wdl be cached 

. fromthecache. ^^^ests the component, .our plugiu should not serve the request • 

The solution is: 

. ' ****** 

the other without . "^^^^^Iw*^ 

* else * 7 ' t / 

out.prlntlnf "another* )/ 

• Add an entry for each TJJil to the CDF. 

.i Use * £ ™^VMinco„co mmmd (a SD pp. OS5atoIlleni , mappedoJU) 
•• the contpou™, UKI p, w . ft a^fe^e,,,.^^ 

err 
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parameters. 



■OH also allows the direct ad coaponcwtaaneM to have diffaert caching 
143 V K ketS777S(pa g e regeneration doesnl do what it says on the tin..) 

14.4 Record-state guid . 

Apage 

processes" the record update fe2^ ( 2? i J , ^ nW C0D?ent w < event 2 )- B * CM 
ignores the SdSSXeSJS rintXf *! plaCenKnt Wilh ° W content < event CM then 
• Apage genera** SSST^ZS^T^tT ^'t * ^ ^ * metada ***°rd state guid. 
accepted (^ventsl-9) ^update (events) yields aplacement (event 7) that is later 



• r 

deployment #n, rcdl 



„J recortSUpdatp (n, rcuSU. 
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... ' 



. 14.5 Clear cache ID 

The following scenario shows how PG generates 
between placement and clear cache. 



a unique clear cache ID to handle a race condition 



.ad 




inst jnd 




cache 


x . 




3 (*) Valid 




7 fx; 


9 y 




3 ly) ?ttvalid 





6 Placemenj^^cld=x) 



Ifccldex) 



• '11 reject 
Placement (ccld«xx) * ^ 
slues Jnd.ccld»y 




Data 


h 2 update 












4- eleS^scache y 








T T 


Oil 


< ■ . ■ ■ 


- r 5 clear cache 












PG2 















tod 

' X 

10 y 




find processes the placeman before the clear each* a!» ~. blMda f i clear <«*»• CM1 receives 
14,6 Instance version 

content tacking IDs and RMS e venfe «m StS^ ™w^» but non-sensjtivjty instance data like 
versions of conC ^ * forsarae inst ^ when generated With different 



The solution is to temporarily keep both .the old and 



new version- of the instance metadata and cache file. 



byappendingi^^^ 
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/cachcRoot/pathAtPg/paramHash_v.sbtml where 
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L^Tto" w A ^ S " ^ <3°«y for an instance cm retain multiple rows taobablv 2 at 

■ ^ti^tZZTt^^™^^ TneverwnisLScZLd 

14.6.1 Regen placement overwrites file 

The mismatch occurs in a window that starts when the cache file fs written with data from content 



deployment *n, rcdl 



1 rocDxdppdatBfn, rcdl) 




ad 




Inst m£ 




caclLQ I 


n-l 
2 n 




n-l Valid 
8 n ' Valid 




n-l 









4 reganfnj 



14,6.2 Normal placement overwrites fiJe 1 

^ Whra caclK ffle is ««"> to to content 

iSteShftrReS^rr" 0 ' 1 "»■*'"""• and PO serve from ,he cache when a* 
version of the cache file ,. written (event 12), so there should not.be a problem in the no regen case. ' 
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deployment 0a, rcdl 



8. recordDpdate(n, rcdl) 




ffd 




inst rod 




cache 


a-1 

9 a" *' 




? n-1 Valid 
3 a " Re^ei 

13. n- <yalid - ( 




4 .a-1 
a? a* 



5^^S«t25^ ^.^Wgtq the instance state transition to " 
placement andves latex, if s ignored, since a Valid instancfalread^te * ^ 
14 J Metadata record locking 

WKnff 0 ^ 8Ceaari °! ' 8h0W ° aSes wh6re ^currency within CMieq 
loclang to ensure correct behavior. ^ 



uires in-memory DB record 



14.7J Record update after CM starts writing cache file. 

metadata table Tthis cas^e Refr^ t t <** to HP*** the instance • 

•(event 1) d«acte^^^^25£? ^ fir5t (even l 6): ^ ^^ce metadata. update 
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deployment #n, rcdl 



4 recordUfcdate (n. 




rs 




n-1 




6 n . 






i 



last md 
gtate 



n-1 invalid 0 
7 n-1 Regan 1 
11 n-1* Invalid 1 
11 n valid 2 



version 



rs o Records rate table 



cache 


8 regen(n) 


n-1 ahcdLO . shtnl * 
5 n-1 abcd£l . ahtml • 
10 n obccL2 . ahtiql 





.'^^ri^^ 2610 ^ 0 ^ and ^taun« the transaction comiits.' 

*,Any transaction reading a row Wocks out writers until the transaction conjmits. 
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record update (event 6): 

SHl3jS^^°^ S " e ^^ S ^r te (3rC<iMt • ^^ta^eia, rc^dt". ff etCds M , 

mdCon. commit « 

write instance metadata (event 7): 

• " Wrr/?^ ld be "derated, 
lastMetadata'. state =• Regen 
else. a 

instMetadata. state = Invalid 
... set other instance metadata fie-Mi 
instMetadata.newJtecord MaaCa fields • v r 
mdc on. commit . 

if lastMetadata. state •== Reflen - 

send regen to WS 
else if iastMetadata. state ~ invalid 

delete cache -.file Anvaiia,^ . , 

14.72 Concurrent placements 

The previous scenario snowed that the metadata nn^ , • ' 

v^etempktein^ 

trueof the ofoer synchronization mecSmSS^S * ^ r ^ cac he file is written. The same is ■ 

following scenario shows- that SC^l^^^ clearCacheld). The 

. ^^^canhappenwith^ 
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inat aid 




r/* state 


version 


3 x&y r 




5 x&y r/w valid 


0 



cache 
4 x&y abcO.shtani 



(event 4). ' (evrat 3) ' 80 both deade to version 0 of the cache file . 



14.7.2.1 Problem l. T concurrent open and write to the 



same file by different threads la the same process 



Here are2 basic option for soM^ • 

L Use file locking to serialize file access 
. 2.. Usesomeothermutualexclus^ 

14.7^2 ■I^l ;l fl^' te rf^^^ lll(llto . 

If the concurrent file access problem is solved «n rtnt j • ' i 

the 1* thread Then both UuLds u^Ln .17? P , <hlBad S1InpIy ovc ™ rite s the file written by 
.1. Jf the failure is "duplicate record," just ignore it. 
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14*7.23 Adding a new state 

with a stSerflS mto t™, i . 2f" * ^"^^SHaiao writea the instance metadata 
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. 14.8 me locking 

14.8.1 V6 Yjs COD file handling 

o Expire without regen: rename cnrlJitml -> curl.html.bak 
■ NT - uses a quite complicated method MTmv* 

• tries MoveFileExW; returns if ok 

• open src file & get a file mapping; share delete, read, & write 
•• -open dst file w/truncate, share delete, read, & write 

• if dst file not found, open w/create new, share delete, read & write 

• for each chunk of src, MapViewOfFile, WriteFile chunk to dst 

• close src file & mapping 

- •• open existing src file, delete on close, share delete, read, & write 

• close src . 

» Other -VUJDS::rename. 
•o Expire with regen: rename temp.html k> curl Jitml '* . 

" wr^s'cThef p^° n * ^^^^^^^Lcpp w/flagthat WS 

• yU_OS^name(tmpKle 1 cunTile,removeSrc=T) 

o . NT - CopyFileW(tmpKle, curlFUe, failIfExists=F)- 

■wunlink(tmpFile) 
o Other - rename ■ 
• •- VU_OS_unlink(curl.html.bak) ■ 
. . " • o NT -._wunlink ' 
' ■ ' • o Other -unlink . \ . . -• 

• cod 

o . Normal placement: 

■ Multiple threads can. concurrently try'to open & write the same file.- Another 
process will not have the file open, because it's a new version 

ZS^^VS^ wiH tousfi java - ni0 fiJe Iockk s when 

a \ ^l»d threadls l?locked ^ "to™ ^sed 2 nd thread's open then 
succeeds &2» d create truncates the file & rewrites it ' ° peninen 

o Regen placement: 

since there will only be J regeti placement- 
o Expire without regen: . 

- Instence deleter thread deletes file. java.io JPile.delcte fails if dst is open bv 
another jvm, but it should still do since deleter doesn't delete instance! till enough 
tune has passed to be assured no one is still using the instance. " 
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U&2 WcWBS^&taMngtoteta, W /PAD,CMD mm |p B I a flngffle S ) 

JUSt " 118 *" was * m 4 - *- "°!«Mly i? woat be 



Original Message 

Prom: Rajkumar, Isaac 
Sent: Friday, January 18, 2002 3; 12 PM 
To: Afshar, Jamshid; Caldwell, David 
■ Subject: RE: ATM and COD 

' ^toZSZEttTlSZ* 0 ^* - » t. <=rae* o £ «, f„ r 

-Original Me s saga . 

Prom: Rajkumar, Isaac -' ■ 

Sent: Friday, January 18, 2002 3:05 fm • * 
To: Afshar, Jamshid; Caldwell, David 
Subject: RE: AFM and COD 

This fix was in 5.0.x and lost' in n «?•.*. \. , * 

is the first time . I am^earSg Sou^ this £ ^ ° f 

occurrence -that is not easily reducible, l^e.this is a pretty rare 

— : — Original Message 

Promt Afshar, Jamshid • 

"Sent: Friday January 18, 2002 3:04 PM V 

To: Caldwell, -David; Rajfcumar, Isaac ■' 
Subject: -RE: AFM and COD 

JiS'^JTS^i^QSaSlSs LrL t0me V S am ?*° What a ^ ea " 'to be the ' 

Mark that QA l« able tfreproLce it * C w iCia) ' * 1 " 0 ' 1 j,ist beard f «» 

. ep i t . 1( u let ^ ^ whafc happens . .. 

Instance version, instance state, and theinstance ckanup thread obviate this problem. 
14.9 WebServerLookup Concurrency 
14.9 J Concurrent placement-msgs 
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use 



14A2 Concnrrenttemptotedeletoinetadata.insganapIacemBit.nisg 

£ n SKiESXf "T l ttmplate ^ 2 thread* can 

Concurrent template add nietadata-msg and placement-msg 

fad ^SSS^SKISi" ?°"' tem P ,ate * fW—f ~=«n, 2 thread* caa 
trtTiQ^ttort ; > u «r«T; cnecjc ror existence of the template metadata in -the same * 

• that adds the templatebcS ^ f ° r *" te * U,te *» 46 Bam * t ™ B *» 

14.10 kedundontmetadata-msgs * * '.' 

1. Add objecfld andmodCoont columns to template metadata ' *' 

3. CMMetadataMsgHandler.addTemplate: *.'''. 

a. if tMd exists, return 

b. add the template md, etc 

4. CM.deleteTemplate: . 

a- if iMd does not exist, return 

b. iftMd.obiectIdr=mdMsg.objectId,retiini ' 

c. delete thetempiate md, etc 
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5. CM.updateTemplate: 

a. if tMd does not exist, return . 

b. -iftMdobjertld^mdMsg.objectld, retum 

c. if tMd.modCount>= m dMsg.m 0 dCount, return . ' . •' 

d. if tMdtemplateHash = mdMsg.templateH^/return " 

e. update template rod, etc 

. Wore the deletion, wien it come. i bSSt^SjS nSlw ^T^v d * " m ^ "°™ 
objectTd, it appears to be a new mesaage 1Z Z * U «"*"• °° 

the template or war was re-added. ™odCount saved m the initedata went bade to 1 when 

- .■ ap P .war created with objectId=in, modCount=l. It contains: . . . • . . . • ' " ' 

a-jsp with bash=a 
b.jsp with hash=b 
"Hiis results in the following metadata: . 

ajsp with objectId=ni, modCountel, hash=a '• 
. b.jsp with objectId=m, modCount=l, hash=b ' ' 
app.war updated with objects, modCount=2. It contains: ' 
. a-jsp with hash=c 

b.jsp with hash=b . 
•This results in the following metadata: • . 

ajsp with objectid=m, modCount=2, hash=c • * f * 
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b.jsp with objectId=m; modCount=l, hash=b 
Now suppose a new AM is added 

app.war is created with objectId=m," modCounfc=2. ■ It contains: 

a. jspwithhash=c 

b. jsp with hash=b ' 

AM sends a metadataMsg containing: 
ajsp with objectId=m, modCount=2, hash=c 
b.jsp with objectldsm, modCount=2, hash=b 
.If CM just looked at objectld and modCount, it wpu]d unnecessarily clear*]! of b.jsp * 



instances. 
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